CloudKnowledge: introduction

The first version is ready for download and testing. See here for installation instructions and a first tutorial.

introduction

Thinking about self-configuring redundant network-storage I figured that there should be versions of the data. And somehow the network has to remember the last version number.

Which led me to the thought of generic cloud knowledge where several servers (don't have to be actual servers) run a small app that stores key-value pairs. These get set and updated via network. And synchronied via network. And clients set and get values via network.

Some more explaination: A client "sets" a value by broadcasting the key and the new value. The servers store this new value. Sometime later a client asks for the value via broadcast. The servers answer by sending key, value and an integer on how old their value is (that way a client can see which value is the newest without the servers having absolute time).

Sometimes also the servers broadcast for answers to see whether their stored value is still correct. And sometimes (especially after start) they ask for all keys to (re-)build their database.

This should allow storage of small or not so small values with redundancy. It doesn't matter if a server closes down unexpectantly, others have the same data. And it doesn't matter if a server is offline for some minutes or hours and doesn't get some updates, the clients see from the answers which one is the best. And servers synchronize automaticly...

installation and short tutorial

If the easy_install tool is installed (in ubuntu its in the "python-setuptools" package), installation is really easy:

pip install http://www.arnoldarts.de/files/cloudknowledge-0.1.tar.gz

Once that is run (preferable as root (Warning! Don't actually do that!)), you can start using the apps that show how it works.

Start cloud-server on one console and watch its output.

Next open another console and set a key value pair with cloud-client -s bla blub. You will see nothing on this console but the console with the server will now contain some (debug-) output that the value got saved.

Now lets ask for the value again with cloud-client -g bla and after ~30 seconds it will output the value it found. [1]

In a second step, install the cloudknowledge on a second computer in your network and make sure both machines firewalls (if activated) allow the port 72374 which is the currently used port for cloudknowledge.

Start the cloud-server on the second machine and watch it synchronize the values with the first server. If you now ask for a keys value, the client will in fact get two answers and pick the newest.

If you now stop the first cloud-server, you will still be able to query the values you had set. Fine, isn't it?

If I did everything right, you can start and stop servers as you like, as long as at least one server is running, the knowledge is not lost.