Infrastructure high frequency data

Typical for « big data », the construction of a high frequency database requires an important storage and data processing capacity. 

The infrastructure forms part of a cloud computing network providing access to the computing and storage capacity of EGI (European Grid Infrastructure) and France Grilles. It will be hosted by the LPSC (Laboratory of Subatomic Physics and Cosmology, a division of the CNRS like EUROFIDAI) in Grenoble.

 
high frequency data storage
NAS and SAN storage

 

The material installed at the LPSC is composed of servers and a specific network (Cisco). Three servers are used for virtual machines under VMWare, another server is used to set up a connection between the global infrastructure of BEDOFIH and the European Grid Infrastructure (EGI). This server uses the iRODs software for distributing data, a software that the LPSC knows very well. Computing servers are also integrated in this structure.

The material for storing the data is composed of 2 sets from EMC² :

  • The first one is a VNX5300 providing a SAN storage (Storage Area Network) with a block-based data access.
  • The second one is a set of 10 Isilon nodes providing a NAS storage (Network Attached Storage) for file type data. Isilon was chosen for its “scale-out” architecture that facilitates incrementing storage volumes without any performance loss, a system that is most convenient for growing data volumes as in the case of BEDOFIH.

The data volume collected each year is estimated to be 100 teraoctets.