Ephemeris Logo

Delivers Blazing-Fast Data Access Speeds

0 Nanoseconds
Per record access speeds
0 Million
Records per second per processor core
Moonshadow’s patented “Ephemeris” database technology powers “in-the-blink-of-an-eye” fast interactive graphical and geospatial data visualization of big datasets in both online and offline environments using desktop, laptop and mobile devices.

Ephemeris is ideal for searching and visualizing big datasets at speeds not previously possible while using fewer servers at lower costs. Ephemeris technology retrieves records from big databases at speeds of approximately 10 nanoseconds per record, making it possible to access 100 million records per second per processor core.

Shrinks Your Big Data Footprint By As Much As 90%

In continuing to improve our patented proprietary datastore “Ephemeris” we recently reached an interesting milestone – we can store big data in less than one byte per value for databases that include a mixture of boolean, numeric and text fields. As an example, one population database we work with is the Registered Voters Database for California from our customer L2 Political.

This database includes 16 million records and 649 columns storing more than 10 billion values. This database includes 211 text fields, a field type that usually uses four bytes per character. This database takes up approximately 92GB in Microsoft SQL Server. Ephemeris stores these 10 billion values in just 8.9GB which comes to less than 1 byte per value. We were able to shrink the footprint by more than 90%.

California Voters Database MS SQL Server Ephemeris
Footprint 91.87 GB 8.90 GB
Records 16,127,412 16,127,412
Fields 649 649
Total # Values (records * fields) 10,466,690,388 10,466,690,388
Average bytes per value 9.42 0.91

Reduces The Number of Servers Required By 90%

Ephemeris can store 10 times as much data on the same server, backup servers can store 10 times the number of records and when moving data around file transfer times are therefore 10 times faster. For some of our customers we have reduced the number of servers they need by hundreds while also increasing the data access speeds. By using Ephemeris they reduced their monthly server costs dramatically.

  • Ephemeris is a no-SQL in-memory software-only solution
  • No specialized hardware is required
  • Ephemeris runs on standard commodity servers and computers
  • Cost savings are enormous vs. traditional database and server solutions
Bit Packing

Most traditional databases round the space they reserve per field up to the next whole byte (or, often, the next power of two in bytes). Ephemeris goes down to the bit level. If we can store a field value in five bits we use just those five bits and we use the remaining three bits for something else. We pack the data very tightly and as a result Ephemeris can store much more data in the same amount of space without doing any compression.

Attribute Clumping

Ephemeris doesn’t store individual cell values per record for some of the fields, instead it stores cell value combinations. In an address database for the US, for example, each ZIP-code will only occur in a small number of cities, one or two counties and in just one state. Instead of storing the ZIP, city, county and state as individual data for each record Ephemeris only stores the occurring combination of values for a record.

Ephemeris Does Not Use Compression

The file format stored on disk is (almost) exactly the same as the file format that we use when we run the data in memory. As a result data loading times are extremely fast because we do not have to do any conversions during file loads. Loading a database is (almost) a straight copy of the data from disk to RAM. The small footprint has resulted in another interesting advantage: access speeds. Because the Ephemeris’ file format is so tiny less memory needs to be accessed to find the records that you need to retrieve. Data access speeds in Ephemeris are in the 10 nanoseconds per record range on standard commodity servers. This means that you can traverse 100 million records per second per CPU core.

Read more about Ephemeris.

Contact Moonshadow  For more information about Ephemeris.