Video Tutorial Object Storage
In this tutorial I propose to discover the principle of Object Storage from a developer point of view. We will not focus here on the technical aspect but rather on the benefits that little we bring this storage system in the context of a web application.
The politics
By default, when you want to store files, the most obvious solution is to use the hard disk of our server. This can work at the beginning of the project but can quickly cause several problems:
- If the application is scalable and more than one server is needed, it will be necessary to find a way to share the disk.
- The disk space is not unlimited and if the application grows it may be necessary to have to buy more storage. Then you have to put in place the necessary policies to distribute the files on the free storage spaces.
- Fault tolerance requires duplicating information in order to be able to quickly switch from one storage to another (which can be done "simply" with a RAID1 system for example, but that poses more problem if you want copies in several parts of the world).
These issues are not simple and require a lot of time; time we prefer to devote to the development of the logic of our application (rather than the technical aspects).
The object storage, the cloud solution
These problems, the web giants have already met, and have created solutions to simplify their management.
Object storage is a solution that simplifies the management of a large amount of data by acting as an intermediary with which we can communicate with a dedicated API. So we add an abstraction layer and we do not have to worry about how the storage is managed internally.
In this system the files are represented by objects that have the following properties:
- a unique identifier
- metadata (which can be used to search for files or store specific information)
- the data corresponding to the file
These objects are saved in an index which then allows to quickly find the associated information (without necessarily having to find or read the file).
Webservices
Even if it is possible to set up an object storage system at home, you will tend to use a third-party service because it offers several advantages:
- The price of storage depends on your consumption.
- The implementation is simplified thanks to the use of widespread interfaces.
- We can choose the type of storage / replication according to the needs (storage "cold" for backup for example).
There is a multitude of services that offer Object Storage and I would not try to make a comparison here (there are many differentiating factors and the choice will depend on your needs)
On the other hand, all the services do not use the same APIs and it will be necessary to adapt its code according to the service used (or to use a library which abstracts the system of file).
Some services rely on the amazon S3 API structure to allow a better interopability (however it will be necessary to pay attention because sometimes all the functionalities of S3 are not reproduced).
Object Storage in PHP
Communication with object storage services in PHP is very simple thanks to layers of abstraction like FlySystem or Wafer. These libraries offer generic methods (get, write) and interfaces with different file systems which allows switching from one system to another by simply changing the Adapter used.
At first we can use the file system of the server.
use League Flysystem Filesystem;
use League Flysystem Adapter Local;
$ adapter = new Local (__ DIR __. '/ uploads /');
$ filesystem = new Filesystem ($ adapter);
// The $ filesystem object will now be used to interact with files
And later replace it with a compatible S3 api or other, without changing the code of our application.
use Aws S3 S3Client;
use League Flysystem AwsS3v3 AwsS3Adapter;
use League Flysystem Filesystem;
$ client = new S3Client ((
'credentials' => (
'key' => '*************',
'secret' => '*************',
)
'region' => 'fr-by',
'version' => 'latest',
));
$ adapter = new AwsS3Adapter ($ client, 'grafikart', 'uploads / path');
$ filesystem = new Filesystem ($ adapter);
