Monday, May 21, 2012

Snappy Compression fails for Apache Cassandra on a Raspberry Pi

Although I’ve managed to get Apache Casssandra running on a Raspberry Pi I’ve been struggling to make any use of it.    I’ve been using the latest build of Cassandra (1.1.0) and when ever I’ve tried creating a column family I’ve been getting the following error:

SnappyCompressor.create() threw an error: org.xerial.snappy.SnappyError [FAILED_TO_LOAD_NATIVE_LIBRARY] null

As you are aware compression on Column Families was introduced in  Cassandra 1 for space saving and increased disk IO (http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-compression).  By default the compression is Snappy using the Google Snappy Library (http://code.google.com/p/snappy-java/ ) which is a wrapper for Google Snappy (http://code.google.com/p/snappy/) written in C++.  There is no port for Google Snappy on the Pi and my initial attempts to build one have not been successful (autogen.sh is failing with undefined macro AC_DEFINE)

You should be able to create a column family with DeflateCompressor from  cqlsh as follows:

create TABLE users (KEY varchar Primary key, password varchar, gender varchar) WITH compression_parameters:sstable_compressor = 'DeflateCompressor';

However that’s failing with the same Snappy Compressor error.   Describing the Keyspace with Cassandra-cli gives the following:
Keyspace: test:
  Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
  Durable Writes: true
    Options: [replication_factor:1]
  Column Families:
    ColumnFamily: users
      Key Validation Class: org.apache.cassandra.db.marshal.UTF8Type
      Default column value validator: org.apache.cassandra.db.marshal.UTF8Type
      Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type
      GC grace seconds: 864000
      Compaction min/max thresholds: 4/32
      Read repair chance: 0.1
      DC Local Read repair chance: 0.0
      Replicate on write: true
      Caching: KEYS_ONLY
      Bloom Filter FP chance: default
      Built indexes: []
      Column Metadata:
        Column Name: password
          Validation Class: org.apache.cassandra.db.marshal.UTF8Type
        Column Name: gender
          Validation Class: org.apache.cassandra.db.marshal.UTF8Type
      Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
      Compression Options:
        sstable_compressor: DeflateCompressor
        sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor

Note the two compressors in action.  I’m not sure quite what’s going on, or if this is the correct result at the moment but I’m continuing to investigate (any answers gratefully received.    So for now it’s back to Apache Cassandra 0.8.10 which doesn’t have compression and seems to work on the Pi (once the env file has been changed).

Update

Thanks to Jonathan Ellis for pointing out the command should be:

Create TABLE users2 (KEY varchar Primary key, password varchar, gender varchar) WITH compression_parameters:sstable_compression = 'DeflateCompressor';

28 comments:

  1. It's showing you the correct option name there, still defaulting to Snappy. The bug is that it allowed you to specify the (ignored) _compressor option as well...

    So, WITH compression_parameters:sstable_compression = '' should disable it, or DeflateCompressor to use deflate.

    I've created https://issues.apache.org/jira/browse/CASSANDRA-4266 to check for unknown parameters.

    ReplyDelete
  2. According to JIRA, CASSANDRA-4266 was fixed in Cassandra 1.1.1.

    ReplyDelete
  3. Cassandra represents one of the most exciting uses of P2P technology to date.

    ReplyDelete
  4. Your blogs and its stuff are so notable and worthwhile it can make me return. great american business products

    ReplyDelete
  5. Great blog you people have made here on this blog site, I will absolutely appreciate your work.
    good infographics

    ReplyDelete
  6. Really amazing blog, I’d love to discover some extra information.
    social media infographics

    ReplyDelete
  7. Congratulations guys, quality information you have given!!!
    infographic design

    ReplyDelete
  8. You have an effect on is totally appreciative and full of interest.social media infographics

    ReplyDelete
  9. This is an informative blog by which I have got that info which I really wanted to get.
    good infographics

    ReplyDelete
  10. I love this blog because it is user friendly with appreciative information. good infographics

    ReplyDelete
  11. congratulations guys, quality information you have given!!! ACLS Renewal

    ReplyDelete
  12. Each time I used to always check blog posts within the first hours in the break of day, because I like to get information increasingly more. comment pirater un compte facebook

    ReplyDelete
  13. I know this is quality based blogs along with other stuff.

    ReplyDelete
  14. The vital information in this blogs has allured me. pirater un compte facebook

    ReplyDelete
  15. Awesome blog! Now In anticipation of a follow-up …. ny criminal attorney

    ReplyDelete
  16. I was searching for many blogs site and now finally I have got this cool place with lots of information.

    free open source ftp client

    ReplyDelete
  17. I have visited lots of the blogs but this blog is truthfully wonderful.vietnam holiday

    ReplyDelete
  18. I am truly appreciative to the owner of this website who has shared this nice article at this wonderful place.
    weight loss supplements for women

    ReplyDelete
  19. Thumbs up guys you are really carrying out a great job.vietnam travel

    ReplyDelete
  20. An enormous round of applause, continue the great work.vietnam tour

    ReplyDelete
  21. Inconceivable points. Sound arguments. Keep going the truly amazing work.vietnam holiday

    ReplyDelete
  22. It’s a classic great for me to go to this blog site, it offers helpful suggestions hair loss

    ReplyDelete
  23. It’s a classic great for me to go to this blog site, it offers helpful suggestions hair loss

    ReplyDelete
  24. Your composing is purely awe-inspiring that I desired to read such high quality material...
    data cabling brisbane

    ReplyDelete
  25. These articles and blogs are truly enough for me for a day. social media infographics

    ReplyDelete