If you manage ‘Big Data’ then you’ve probably run into this problem: your Big Data is getting too big. Before you know it you are moving around files that take 20 or 30 hours to transfer and many hours to compress, decompress or even to back up. Six months after installing a new batch of backup servers they are maxed out and you have to add more. As a result you are constantly pressing users to reduce the amount of data you keep accessible for them. At Moonshadow we host and serve more than one billion records for our customers and we used to run into this problem constantly. A few years ago we came up with a simple idea to solve this problem: if we could shrink the footprint of each record we could reduce the size of the data. And if we could do this in the native format of the datastore then we would not have to deal with compression and decompression.
Over the last five years we have been developing our patent-pending proprietary datastore “Ephemeris” and recently reached an interesting milestone: we can store big data in less than one byte per value for databases that include a mixture of boolean, numeric and text fields. As an example, one population database we work with is the Registered Voters Database for California from our customer L2 Political. This database includes 16 million records and 649 columns storing more than 10 billion values. This database includes 211 text fields, a field type that usually uses four bytes per character. This database takes up approximately 92GB in Microsoft SQL Server. Ephemeris stores these 10 billion values in just 8.9GB which comes to less than 1 byte per value. We were able to shrink the footprint by more than 90%. As a result we can fit 10 times as much data on the same server, our backup servers can store 10 times the number of records and when we move data around file transfer times are 10 times faster. In another example we have a database with 160 million records and 170 fields or a total of 27 billion values. Ephemeris takes just 16GB to store this data which comes to 0.63 bytes per value. This is smaller than the space that is taken up by the smallest datatype in a traditional SQL Server database. Using Ephemeris some of our customers reduced the number of servers required from several hundred to less than 90, resulting in savings of tens of thousands of dollars per month.
Table 1. Ephemeris reduced the registered voters database for California by 90% (Data from L2 Political)
|California Voters Database || MS SQL Server || Ephemeris |
|Footprint (GB) ||91.87 ||8.90 |
| Records ||16,127,412 ||16,127,412 |
| Fields ||649||649|
|Total Number of values (records * fields) ||10,466,690,388 ||10,466,690,388 |
| Average bytes per value ||9.42 ||0.91 |
[click to continue…]
As you probably know Ground Game with VoterMapping is the most technologically advanced suite of tools for online voter data analysis and door-to-door canvassing with iOS (iPads and iPhones) and Android (tablets and phones) mobile devices.
We’ve also added a new desktop web-browser interface to enable phone banking. Your canvassing volunteers can login from their laptops or home computers to conduct phone surveys! The latest improvements to Ground Game Web include a split screen allowing your phone banking volunteers to see a map of the addresses and voters they are contacting while navigating through the call list and survey questions.
Phone banking is among the most efficient ways to talk with community members and prospective voters about the issues that matter to them and to collect actionable survey responses. Using Ground Game with VoterMapping campaign management staff and field coordinators simply use the selection tools in VoterMapping to easily create targeted lists of voters with phone numbers, save those lists and assign them to phone volunteers with a few mouse clicks. Simple, fast and efficient! Volunteers login to the Ground Game Web interface from their computer’s web browser, select voters from the list, call them, conduct the campaign-generated survey, and the responses are then uploaded in real time to the campaign’s VoterMapping management interface.
A phone bank can remind and motivate targeted voters to vote, advise them on voting locations, offer rides to the polls or introduce a candidate’s positions on the issues. The call list should target the correct audience; for example, a GOTV pone bank should concentrate only on your candidate’s likely voters. With VoterMapping’s best-of-breed registered voter data and hundreds of voter attributes it’s easy to drill down to create exactly the targeted audience that meets your campaign’s strategic goals.
Learn more about Ground Game with VoterMapping
Residential Solar Installation is a new and emerging market. Like in all markets it takes a large amount of time and financial resources for businesses to find and connect with potential customers. Ground Game can help streamline your sales process by providing a steady flow of information and a set of organizational tools vital for running any door to door operation. The increase in efficiency can easily be worth tens of thousands of dollars or more in worker hours.
Ground Game is equipped with an extensive commercial database that contains well over 220 million adults across the United States. You will be able to filter and select people from over 200 demographic, economic, and social attributes such as Estimated Income, Property Type, Home and Land Values, Homeowner Status, Occupation and much more. This data is displayed over Bing Maps, allowing you to easily visualize where your target clientele is located. These selections can then be broken down into smaller, geographical based sections, to create walking lists for your door to door sales operations.
Before your sales team even steps foot into the field, they will be able to use the online canvassing platform to view and tag addresses based on site suitability.
Armed with smartphones and tablets, your sales team can download their assigned walking lists directly onto their device, and go into the field to collect valuable information with the Ground Game mobile app. The information they collect is seamlessly sent back to the database and accessed from the mapping interface. From here you will be able to track your canvassers progress, conduct further analysis, or export the collected data to a csv file to feed into your CRM, further enhancing your work-flow.
Contact Moonshadow for more information or to schedule a demo of Ground Game.
Ground Game adds sophisticated tracking tools
With Ground Game you always know where your field teams are. Ground Game is an easy-to-use mobile app with a customizable questionnaire for your sales reps to qualify leads on iPads and Android Tablets.
Ground Game shows the location of dozens of Sales Reps in Real-Time
[click to continue…]
Ground Game with VoterMapping has a new new route planning tool that makes it easy for field managers to define specific routes for their walking lists for mobile canvassing. Simply zoom in on the map (using either road or satellite view) in the VoterMapping management interface and mouse over the address markers to define a specific walking route sequence for your mobile canvassers.
Once you’ve defined the route plan, assign the routed walking list to one or more of your mobile canvassers in the field using the canvasser-assignment controls in VoterMapping. Your canvassing volunteers will then see the routed walking list on their mobile devices with the addresses appearing in the specific routing sequence you defined.
You can further fine tune the sequence of the addresses and route plan (after mousing over them on the map) by using the address grid in the right side panel. Delete specific addresses or change the order by dragging a specific address up or down on the list.
Learn more about Ground Game with VoterMapping