[ACCEPTED]-Serialize Data Structures in C-serialization

Check out tpl. From the overview:

Tpl is a 7 library for serializing C data. The data 6 is stored in its natural binary form. The 5 API is small and tries to stay "out 4 of the way". Compared to using XML, tpl 3 is faster and easier to use in C programs. Tpl can 2 serialize many C data types, including 1 structures.

I know you're asking for a library. If you 40 can't find one (::boggle::, you'd think 39 this was a solved problem!), here is an 38 outline for a solution:

You should be able 37 to write a code generator[1] to serialize 36 trees/graphs without (run-time) pre-processing 35 fairly simply.

You'll need to parse the 34 node structure (typedef handling?), and write the 33 included data values in a straight ahead 32 fashion, but treat the pointers with some 31 care.

• For pointer to other objects (i.e. char *name;) which 30 you know are singly referenced, you can serialize 29 the target data directly.

• For objects that 28 might be multiply refernced and for other 27 nodes of your tree you'll have to represent 26 the pointer structure. Each object gets 25 assigned a serialization number, which is 24 what is written out in-place of the pointer. Maintain 23 a translation structure between current 22 memory position and serialization number. On 21 encountering a pointer, see if it is already 20 assigned a number, if not, give it one and 19 queue that object up for serialization.

Reading 18 back also requires a node-#/memory-location 17 translation step, and might be easier to 16 do in two passes: regenerate the nodes with 15 the node numbers in the pointer slots (bad 14 pointer, be warned) to find out where each 13 node gets put, then walk the structure again 12 fixing the pointers.

I don't know anything 11 about tpl, but you might be able to piggy-back 10 on it.

The on-disk/network format should 9 probably be framed with some type information. You'll 8 need a name-mangling scheme.

[1] ROOT uses this 7 mechanism to provide very flexible serialization 6 support in C++.

Late addition: It occurs to me that this 5 is not always as easy as I implied above. Consider 4 the following (contrived and badly designed) declaration:

enum {
   mask_none = 0x00,
   mask_something = 0x01,
   mask_another = 0x02,
   /* ... */
   mask_all = 0xff
};
typedef struct mask_map {
   int mask_val;
   char *mask_name;
} mask_map_t;
mask_map_t mask_list[] = {
   {mask_something, "mask_something"},
   {mask_another, "mask_another"},
   /* ... */
};
struct saved_setup {
   char* name;
   /* various configuration data */
   char* mask_name;
   /* ... */
};

and 3 assume that we initalize out struct saved_setup items so that 2 mask_name points at mask_list[foo].mask_name.

When we go to serialize the 1 data, what do we do with struct saved_setup.mask_name?

You will need to take care in designing your data structures and/or bring some case-specific intelligence to the serialization process.

This is my solution. It uses my own implementation 14 of malloc, free and mmap, munmap system 13 calls. Follow the given example codes. Ref: http://amscata.blogspot.com/2013/02/serialize-your-memory.html

In 12 my approach I create a char array as my 11 own RAM space. Then there are functions 10 for allocate the memory and free them. After 9 creating the data structure, by using mmap, I 8 write the char array to a file.

Whenever 7 you want to load it back to the memory there 6 is a function which used munmap to put the data 5 structure again to the char array. Since 4 it has virtual addresses for your pointers, you 3 can re use your data structure. That means, you 2 can create data structure, save it, load 1 it, again edit it, and save it again.

You can take a look on eet. A library of the 6 enlightenment project to store C data types 5 (including nested structures). Although 4 nearly all libs of the enlightenment project 3 are in pre-alpha state, eet is already released. I'm 2 not sure, however, if it can handle circular 1 references. Probably not.

http://s11n.net/c11n/

HTH

0

you should checkout gwlib. the serializer/deserializer 2 is extensive. and there are extensive tests 1 available to look at. http://gwlib.com/

I'm assuming you are talking about storing 11 a graph structure, if not then disregard...

If 10 your storing a graph, I personally think 9 the best idea would be implementing a function 8 that converts your graph into an adjacency 7 matrix. You can then make a function that 6 converts an adjacency matrix to your graph 5 data structure.

This has three benefits (that 4 may or may not matter in your application):

• adjacency matrix are a very natural way to create and store a graph
• You can create an adjacency matrix and import them into your applications
• You can store and read your data in a meaningful way.

I 3 used this method during a CS project and 2 is definitely how I would do it again.

You 1 can read more about adjacency matrix here: http://en.wikipedia.org/wiki/Modified_adjacency_matrix

Another option is Avro C, an implementation of 1 Apache Avro in C.

Here is an example using the Binn library (my 5 creation):

  binn *obj;

  // create a new object
  obj = binn_object();

  // add values to it
  binn_object_set_int32(obj, "id", 123);
  binn_object_set_str(obj, "name", "Samsung Galaxy Charger");
  binn_object_set_double(obj, "price", 12.50);
  binn_object_set_blob(obj, "picture", picptr, piclen);

  // send over the network
  send(sock, binn_ptr(obj), binn_size(obj));

  // release the buffer
  binn_free(obj);

If you don't want to use strings 4 as keys you can use a binn_map which uses 3 integers as keys.

There is also support 2 for lists, and all these structures can 1 be nested:

  binn *list;

  // create a new list
  list = binn_list();

  // add values to it
  binn_list_add_int32(list, 123);
  binn_list_add_double(list, 2.50);

  // add the list to the object
  binn_object_set_list(obj, "items", list);

  // or add the object to the list
  binn_list_add_object(list, obj);

In theory YAML should do what you want http://code.google.com/p/yaml-cpp/

Please 1 let me know if it works for you.