Skip to content

crc32 is overridden by another library when using bundler #38

Open

Description

@n-nishizawa

I inserted the same input, but I got different bitmap on Linux and Mac.

I ran the following code.

  • test.rb
require 'cbloomfilter'

bf = CBloomFilter.new(16, 1, 7, 1)

puts "m #{bf.m}"
puts "k #{bf.k}"
puts "b #{bf.b}"

bf.insert 'a'
puts "bitmap: #{bf}"
  • Result on Linux
$ bundle exec ruby test.rb
m 16
k 1
b 1
bitmap: 1000000000000000
  • Result on Mac
m 16
k 1
b 1
bitmap: 0000000000000100

I added rb_warn into crc32 to check if the function crc32 in your gem is called.


... snip ...

#include "crc32.h"

unsigned int crc32(unsigned int crc, char *buf, int len) {
    while (len > 0) {
        crc = crc_table[(crc ^ *buf) & 0xff] ^ (crc >> 8);
        rb_warn("crc: %d\n", crc);
        --len;
        ++buf;
    }
    return crc;
}

And I ran test.rb with bundler, but I got no warning output by rb_warn.
From this result, maybe crc32 in your gem is not called.

I also ran test.rb without bundler specifying LOAD_PATH explicitly.

$ ruby -I./vendor/bundle/ruby/2.2.0/bundler/gems/bloomfilter-rb-8c9da9516796/ext/cbloomfilter test.rb
m 16
k 1
b 1
test.rb:9: warning: crc: -1529756563

bitmap: 0000000000000100

The waning output came out and I got the same bitmap as that on Mac.

So crc32 seems to be overridden by another library when using bundler on Linux.

Activity

  1. igrigorik commented on Jul 26, 2015

    @igrigorik
    Owner

    Huh, well that's a fun one. Thanks for tracking this down... /me scratches head, not quite sure what's going here.

    I guess we could/should the built-in method: http://ruby-doc.org/stdlib-1.9.3/libdoc/zlib/rdoc/Zlib.html#method-c-crc32

  2. n-nishizawa commented on Jul 28, 2015

    @n-nishizawa
    Author

    Thank you so much for your reply!!

    I have no idea why this strange thing happens too...

    In my investigation, Zlib.crc32 might be used on Linux with bundler.
    Because the index based on Zlib.crc32 corresponds to this gems's bit field.

    [6] pry(main)> require 'cbloomfilter'
    => true
    [7] pry(main)>
    [8] pry(main)>
    [9] pry(main)>
    [10] pry(main)> bf = CBloomFilter.new(16, 1, 7, 1)
    => #<CBloomFilter:0x007f66e7290cc0 @hash_value={}>
    [11] pry(main)>
    [12] pry(main)>
    [13] pry(main)>
    [14] pry(main)>
    [15] pry(main)> bf.insert 'a'
    => nil
    [16] pry(main)>
    [17] pry(main)> bf.to_s
    => "1000000000000000"
    [18] pry(main)>
    [19] pry(main)>
    [20] pry(main)> Zlib.crc32('a', 7) % 16
    => 0
    [21] pry(main)>
    [22] pry(main)> bf.insert 'b'
    => nil
    [23] pry(main)>
    [24] pry(main)> bf.to_s
    => "1000000000100000"
    [25] pry(main)>
    [26] pry(main)> Zlib.crc32('b', 7) % 16
    => 10
    [27] pry(main)>
    

    To fix it, I think there are two options.
    One is to rename crc32 in your gem to prevent the name conflict.
    Another is to use builtin crc32 such as Zlib.crc32 like your suggestion.

  3. igrigorik commented on Aug 2, 2015

    @igrigorik
    Owner

    We should update the code to use the provided crc32 implementation.. no reason to duplicate it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions