What technologies for the site?


In this article, I suggest you share with you the technical choices I was able to make for Grafikart.fr and the reasons behind these choices.

General architecture

There are now different techniques to create a website and before you start talking about language / technologies, you have to make a choice on the approach to adopt.

SPA, single page application

The SPA approach is quite unsuitable for grafikart because there are many entry pages and many users end up visiting only one page. Also, a large part of the traffic comes from search engines and SEO cannot be neglected.

Static Rendering

The principle of static rendering is to render HTML files for the different pages of the site. Rendering can be done automatically using data from an API or database. This type of rendering could have been possible on certain pages (such as tutorial pages for example) but the frequency of updates, and the number of pages to generate would have made the process too complex.

SSR, server-side rendering

For this type of rendering, the server will dynamically generate the HTML pages according to the user request. The server will be able to communicate with various private (database, internal & external APIs, etc.) or public services in order to retrieve the data needed to display the page.

The problem with this type of rendering is that each page has to be fully reloaded when the user clicks on a link. This can be problematic when you want to persist things between pages (such as a connection to websockets for example).

Hybrid, SPA + SSR

A final approach is possible by mixing the rendering on the server side (or static) with the rendering on the client side of an SPA. The objective is to use the same code on the client side and on the server side to have an isomorphic rendering. During a user request, the page is generated and the HTML rendering is returned by the server. JavaScript is then loaded from above and replaces the structure of the page (without visual change for the user). The rest of the navigation is then done on the client side and allows to have the benefits of the client side rendering (speed and fluidity) without the disadvantages (longer initial loading and poor referencing).

However, for me this approach has a few drawbacks:

  • The use of a common technology between the back-end and the front-end makes the system more sensitive to changes (the front-end tends to evolve faster than the back-end).
  • The complexity of the project is more important because it is necessary to develop an API on one side and the client rendering on the other (this can also be seen as an advantage because it brings better separation of the code and we can keep the 2 parts separate ).

My choice: SSR

I therefore chose to render a classic server-side rendering because it is a system with which I am comfortable but also because the site has only one possible format (HTML rendering) and I therefore did not have necessarily benefit to design an API. The hybrid side, although interesting, did not convince me because requires a certain technology and the available options lack flexibility.

What language for the back-end?

Now that the choice has been made for the approach, we must choose the language that will be used to generate the pages on the server side. I did not choose among all the available languages ​​but only among the ones I knew (so if you don't see your favorite language in the list it's not because it's a bad language).

Golang

Golang is an interesting language because it can be compiled to run on different systems, it has a well-structured typing system and has a large standard library. On the other hand, it is a language which is quite technical and which, in my opinion, is more interesting for lower level systems than a simple internal site with HTML rendering.

Elixir (with phoenix)

Elixir is a dynamic functional language based on the Erlang virtual machine. It also has a web framework, Phoenix, which is very interesting with a "Live View" system which allows updates to be pushed from the server to the client.

Unfortunately, the ecosystem is still relatively young (for the elixir part) and we must also quickly put our fingers on the system on which the language is based (understand the erlang language and its virtual machine).

Ruby (with Ruby on Rails)

The previous version of the site was developed using Ruby and the Ruby on Rails framework (I was a bit bored with PHP at the time and wanted to see if the grass was greener elsewhere). Over time, and with practice, the dynamic side of language has proven to be problematic.

For example, it is possible in Ruby to declare the same class several times, which has the effect of adding methods.

class Integer

    # We add a double method to all integers
    double def
        self * 2
    end

end

puts 3.double # 6

This approach initially appealed to me but in use it quickly becomes difficult to know the methods that are available on a particular object and it no longer suits me (tools like Sorbet can however mitigate the problem).

Apart from this aspect (which is a personal preference) the language remains very pleasant and I definitely lose writing comfort by choosing another language.

NodeJS

The choice of NodeJS did not attract me too much because I do not necessarily have a very great affinity with the JavaScript language. Beyond that, the technology is undergoing many changes and I do not find that this makes it a solid base for an application that must last over time.

The ecosystem is also evolving very quickly and I find that exploring the source code of libraries is made too difficult by the multitude of tools used to generate the code (the output source code is difficult to explore and modify via npm link ).

PHP

PHP is the language with which I have the most experience that I had given up a little before the release of version 7. Since then, the appearance of typing (even if it is rather poor compared to other languages) Reconciled me with the language because it brings more structure and improves the readability of the code. This new functionality was also quickly adopted by the community (most methods from third-party libraries are typed and it is easy to predict the available methods and the types of return).

The language is also well installed and has stable monolithic frameworks that match my need for generating HTML pages.

Which framework?

So I chose PHP for this new version and that brings me to a second choice: Laravel or Symfony.

Laravel

Laravel is a framework that I like for the same reasons as ruby ​​on rails but its dynamic side makes it more difficult to analyze code. For example, it is not possible to know the shape of an object coming from the database.

class Post extends Eloquent {

}

$ post = Post :: find (1);
$ post-> id; // 3

Laravel relies on magic methods for accessing data, which is much less explicit than properties defined in code directly.

class Post {

    public int $ id;

}

Also, the framework evolves quite quickly with fairly frequent breaking changes (without necessarily a depreciation phase) which can make maintenance quite complex (a major number change every 6 months).

Symfony

Symfony is the opposite with a configuration-based approach which tends to be very verbose (here is an example of a model). This verbosity however allows a better transparency of the operation of the different classes that make up our application.

On the other hand, the symfony components are less standardized than Laravel and their configuration can be quite difficult because it is not necessarily easy to know all the available options and the effect that this can have on the functioning of the bundle ( we will quickly have to dig into the source code, which is often uncommented, to understand the role of an option).

Laravel or Symfony?

Finally, I made the choice of stability for this project and since I wanted to set up a specific organization, I said to myself that Symfony would be more suitable (to see over time if this choice was the right one ^^) . You can find the discussion that led to this choice on github issues.

The database

The old site worked with a MySQL database and a Neo4J database. The choice of Neo4J was made to allow me to better define the relationship between a course and a technology. The idea was ultimately to be able to create an intelligent recommendation system based on videos marked as read by users. However, in use the system did not really work and I preferred to move towards a more manual system with courses organized in the form of courses (nothing beats human curation).

No NoSQL for the main database

As with the choice of languages, the lack of rigor systematically leads me to organizational problems. Relational databases force me to think about my structure upstream and are often less specialized in the type of data they can handle. However, this does not prevent me from using Redis for the storage of certain information such as user sessions, cache and instant notifications.

Why PostgreSQL?

On my projects I find myself a little lost on the different storage engines of MySL and MariaDB and following the evolution of these 2 databases has become a little too complex for my taste. Also, I thought that with PostgreSQL it would be easier to predict the features that are available on the system (only the version number counts and only one storage system). I also wanted to use PostgreSQL to manage the search part of the forum and the search types seemed more suited to my problem (management of the language and weights to be applied to the different columns).

The research

For research I was using Elasticsearch on the old site but its complexity, and my inability to configure it correctly, meant that the research was not necessarily relevant. Also, for this new version I decided to try other tools.

PostgreSQL

PostgreSQL has a full text search system that can be used to perform a search with relevance ranking.

This is for example what I use to index the topics on the forum.

ALTER TABLE forum_topic ADD search_vector tsvector DEFAULT NULL;
CREATE INDEX search_idx ON forum_topic USING GIN (search_vector);
- When a topic is updated, we update the full text field
-- Function
CREATE FUNCTION update_forum_document () RETURNS trigger AS $$
begin
    new.search_vector: =
    setweight (to_tsvector ('french', coalesce (new.name, '')), 'A')
    || setweight (to_tsvector ('french', coalesce (new.content, '')), 'B');
    return new;
end
$$ LANGUAGE plpgsql;
- The trigger
CREATE TRIGGER update_forum_document_trigger
BEFORE INSERT OR UPDATE ON forum_topic FOR EACH ROW EXECUTE PROCEDURE update_forum_document ()
- We update the initial data
UPDATE forum_topic
SET search_vector = setweight (to_tsvector ('french', name), 'A') || setweight (to_tsvector ('french', content), 'B')
WHERE 1 = 1;

Concretely, this approach works but I find it quite verbose and complex to set up (I am not very comfortable with functions and triggers). The other problem, compared to ElasticSearch, is that the weight of the fields must be defined upstream (at insertion) rather than at the time of the search.

Typesense

Typesense is a simpler tool with which it is possible to interact with the index through an HTTP API. It's much more limited in terms of functionality but fits my use case well.

// We create the index in HTTP (POST)
$ this-> client-> post ('collections', (
    'name' => 'content',
    'fields' => (
        ('name' => 'title', 'type' => 'string'),
        ('name' => 'content', 'type' => 'string'),
        ('name' => 'category', 'type' => 'string ()'),
        ('name' => 'type', 'type' => 'string', 'facet' => true),
        ('name' => 'created_at', 'type' => 'int32'),
        ('name' => 'url', 'type' => 'string'),
    ),
    'default_sorting_field' => 'created_at',
));

// We can then index a new content
$ this-> client-> post ('collections / content / documents', $ data);
// And then search
$ this-> client-> get ('collections / content / documents / search? q = php & per_page = 10 & query_by = title, category, content')

The system works well and is above all extremely fast. On the other hand, it sometimes gives results that are a little too broad because of its correction of typographical errors. For example a search for "event" will bring out "come back" or "event" before other results which would contain event directly.

Instant notifications

For the instant notification system on the site, I chose to use the Mercure protocol. I have had the opportunity to use it several times and I find its operation relatively simple and it adapts with many solutions thanks to its HTTP API.

The front-end

Now that we have covered the technologies that make it possible to run pages on the server side, we will see the technologies that are used on the front-end.

CSS house

Even if some classes on the site may make you think of certain CSS frameworks, I didn't use any for the following reasons:

  • Frameworks like bootstrap offer a collection of pre-styled elements to quickly build an application prototype. In my case, I made a mockup upstream and it would have been counterproductive to use a framework of this type and then have to overwrite the styles offered by default.
  • Utility frameworks like tailwindCSS offer thousands of classes that must be combined to style items. Personally, I don't like this approach because it makes the HTML code hard to read and you have to use tools like purgeCSS to keep only the CSS actually used.
  • I've been doing CSS for a long time and I know how to organize my code to find my way around it properly. The waterfall does not scare me ^^

The approach used by the site is a mix between utility (by grouping together so as not to have 10 classes per element and by using CSS variables to manage variants) and semantics. You can also have an overview of utility classes on this page).

CustomElement & Preact

The server is responsible for returning the majority of the HTML code but some elements need to be dynamic and in this case it is necessary to use JavaScript. The main problem is then to know how to attach the behavior to an element while being able to do it for new incoming elements (in the case of an addition following an AJAX request for example).

Regarding this problem, I decided to use custom-elements which allow you to declare custom HTML elements. I use these web components to create interactive elements within a server generated page.


    
        See the video

This approach also makes it possible to display elements by default while the JavaScript is loading and to avoid changes to the structure of the page as much as possible.

To create some of these components I used Preact. This allows me to simplify the logic and avoid too many manipulations at the DOM level. I chose Preact because it is lightweight and uses the widely supported jsx syntax.

Finally, for navigation I use Turbolinks which will transform all the links on the site into Ajax links. This allows me to maintain the connection to the notification server while the user is browsing but also not having to rerun the entire JavaScript initialization step on each page.

Accommodation

Finally, the last choice to be made is in terms of accommodation: Web service or Dedicated server ?

From the start I ruled out the web services solution because of a high operating cost (especially for the video broadcasting part) but also for fear of being dependent on a particular service. Indeed, one of the problems I have with many current cloud solutions is that each vendor has their own products (each with specific terminology) that require significant prior learning. And the knowledge acquired on the operation of a service is not necessarily transferable to another system.

Also, the size of the project makes it possible to manage it quite easily on a single machine and I preferred to choose a machine on which I have control over the configuration.

A cloud virtual server

So I chose to host this new version on a virtual server. The objective is to be able to change the configuration according to the needs in terms of RAM and storage.

For the host, I hesitated among 2 hosts that I know well: Scaleway (where I use virtual intances during development and object storage) and Infomaniak (which I use to host certain WordPress and for manage my domain name). I finally decided for Infomaniak because they offered more predictable upstream pricing.

After contacting them, they agreed to sponsor this new version by offering hosting for this new version ^^.

To sum up

You now know everything about the choices I was able to make for the site so here is a little summary of the stack:

  • Virtual dedicated server with Debian
  • Nginx for the web server
  • PHP with Symfony for the backend
  • PostgreSQL for the database
  • Redis for cache, sessions and queue system
  • Mercury for instant notifications
  • Messenger for asynchronous tasks
  • Typesense for research
  • NodeJS for the discord bot

RELATED ARTICLESMORE FROM AUTHOR