Creating Online Scholarly Articles Database

How it started

The company Cosmolab came to us with the task to develop a scholarly articles database for storing academic articles. They decided to arrange articles using a web portal with a tree structure. This seemed to be the easiest and most convenient to read and comment on texts, search for required data, and assign different levels of reliability.

The tree structure of academic articles is complicated to code but comfortable to use. We met the client requirements and began working on the project.

Solution

We analyzed the requirements, prepared a detailed technical specification with mockups, discussed with the client what tech stack is better to use for the project. First, we planned to work on Drupal as it supports different versions of articles and translations. After we delved deeper into the documentation, we preferred the framework Yii2 as it proved to work faster than Drupal, use fewer resources, and be simple to support. Then we provided the final project estimation and focused on two directions:

• to build a tree structure for articles
• to segment users and assign various features for every segment

The tree structure for articles was the highest priority, and we started development right from the tree view.

How we built a tree structure

We needed to create a tree structure for academic article storage where:

• every article has only one parent
• every article has from zero to infinity children
• the article hierarchy includes as many nodes as possible

We studied 4 patterns to work with the trees:

• Adjacency list
  
  It’s simple to add an ancestor node ID to the data table.
  Pluses: descendants inherit from ancestors easily.
  Minuses: you cannot get the full articles tree structure and pave the way from the tree root to a specific node.

• Nested SetsYou can add three fields simultaneously to the data table: ancestor ID, descendant ID, and level of the node.
  Pluses: Good for storing straightforward tree models.
  Minuses: Complicated to move and delete separate nodes.

• Materialized PathYou put the text line consisting of the whole route — from the tree root to the specific node — into the data table.
  Pluses: Quickly to choose a tree part.
  Minuses: Complicated to move and delete tree nodes.

• Closure Table
  You create a specific table to store a current record ID and a record ID of the direct descendant. In fact, it’s one of the Adjacency List options but using a third-party table.

We weighed up the pros and cons and chose to use a Materialized Path as it was perfectly suited for our requirements. To implement the pattern we used Itree — a PostgreSQL model that speeds up and eases the work with a hierarchy structure. To draw a tree in 2D we applied the library Viz.js with simple settings and the opportunity to quickly draw up to 2000 tree nodes.

While working over the tree structure, we faced an issue: how many nesting levels should be shown in the window to keep the data readable? Finally, we made 4 nesting levels. Then we built standard components to filter the articles on the web portal:

• Name of the academic article
• The Author: Name and Scientific Degree
• Creation date and Latest text corrections
• The text of the academic article
• Comments
• The article’s place in the tree hierarchy

How we segmented users

We divided the web portal users into 3 segments. Each of them had their own access level.

• Readers — non-authorized users. They can search and read articles using the web portal. They can send a request to be an author.
• Authors/Editors — authorizing through a login and password. They can comment, add and edit materials below the node they have access for.
• Administrators — authorized users. They manage the whole system: add, edit, and delete articles across the tree, moderate authors lists.

To segment the users we adopt the RBAC approach — Role Based Access Control Approach from the Yii2.

The Outcome

A web portal for academic articles storage is a non-standard project. We had to try several alternative libraries to find the best way to work with a tree structure. We used Cytoscape.js, Sigma js, Alchemy js, D3.js and learned new technologies: Ltree, a PostgreSQL module to store articles and Viz.js library to draw a 2D tree. To control the budget, we analyzed, discussed with the customer and deleted some excessive features from the project: tags for articles, the search history, FAQ.

At the same time, we realized the basic functionality in full and as detailed as we could. The web portal supports two languages: English and Russian.

Web portal users are able to:

• register and sign in to the system
• look through articles and edit them
• search for materials using keywords
• create article reviews — posts with common topics
• add a scientific basis

We are proud of the result we got. Now Cosmolab fills out the tree structure with content to provide scientific organizations and institutes with a user-friendly tool for arranging articles.

Stack

PostgreSQL, PHP 7.2, Yii2, Bootstrap, tree, Viz.js, Sphinx