Monday, April 06, 2015

data serialization: Google FlatBuffers

Interesting and very efficient data serialization library:

Google Developers Blog: FlatBuffers 1.1: a memory-efficient serialization library

FlatBuffers: Main Page
"FlatBuffers is an efficient cross platform serialization library for C++, with support for Java, C# and Go. It was created at Google specifically for game development and other performance-critical applications. Why use FlatBuffers?
  • Access to serialized data without parsing/unpacking - What sets FlatBuffers apart is that it represents hierarchical data in a flat binary buffer in such a way that it can still be accessed directly without parsing/unpacking, while also still supporting data structure evolution (forwards/backwards compatibility).
  • Memory efficiency and speed - The only memory needed to access your data is that of the buffer. It requires 0 additional allocations. FlatBuffers is also very suitable for use with mmap (or streaming), requiring only part of the buffer to be in memory. Access is close to the speed of raw struct access with only one extra indirection (a kind of vtable) to allow for format evolution and optional fields. It is aimed at projects where spending time and space (many memory allocations) to be able to access or construct serialized data is undesirable, such as in games or any other performance sensitive applications. See the benchmarks for details."
Why not use Protocol Buffers, or .. ?"Protocol Buffers is indeed relatively similar to FlatBuffers, with the primary difference being that FlatBuffers does not need a parsing/ unpacking step to a secondary representation before you can access data, often coupled with per-object memory allocation. The code is an order of magnitude bigger, too. Protocol Buffers has neither optional text import/export nor schema language features like unions."

Cap'n Proto: Cap'n Proto, FlatBuffers, and SBE


This could be quite useful for efficient IoT communication...

startups: playground.global by Andy Rubin

Android Creator Andy Rubin Launching Playground Global - WSJ
"Playground won’t invest in startups, but will take equity stakes in return for its support.
Playground also plans to help entrepreneurs with distribution, manufacturing, financing and ways to integrate their devices with remote computing resources known as the cloud.
This approach is common in Silicon Valley and is known as an incubator or accelerator. Mr. Rubin described Playground as a “studio,” where inventors, tinkerers and entrepreneurs can focus on building new gadgets and not worry about other aspects of running a business."

...
Our aim is to free the creators to create,” he said in an interview. “By bringing these partners to the table we can remove many of the roadblocks of bringing a great idea to market.”
...
H-P, which sells computers and printers in more than 100 countries, will help Playground startups distribute their products globally, Mr. Rubin said. Google, Tencent and Seagate will help startups work with the cloud, he said. Redpoint will advise startups on financing and may invest in companies that emerge from Playground"



Playground Global (http://playground.global/)

The name is far from unique, but the domain name is interesting, ".global" TLD