amq_server segfault
Forum » Ongoing Development / Problem reports » amq_server segfault
started by: evil_ottoevil_otto
on: 1175125570|%e %b %Y, %H:%M %Z|agohover
number of posts: 4
rss icon RSS: new posts
summary:
Using my own client library (in development), I can reliably cause OpenAMQ/1.2c1 to segfault. There is a good possibility that my library is buggy or non-conforming, but the server must not crash, even when handed a bad data stream.
unfold all | fold all | +more options
fold
amq_server segfault
evil_ottoevil_otto 1175125570|%e %b %Y, %H:%M %Z|agohover

I have 2 client programs, a sender and a receiver.

The sender connects, opens a channel, and calls basic:publish to send a number of messages to amq.direct.

The receiver connects, opens a channel, declares a queue, binds the queue to amq.direct, calls basic:consume, and sits listening for incoming messages.

If I start my receiver, abort it (hit control-c to cause an abnormal disconnect), run my sender, and then start my receiver again, openamq segfaults.

(I hope to have my client in distributable form shortly, currently it is just a pile of files)

OS is linux 2.4.

Here is trace output:

$ ./amq_server -v
OpenAMQ/1.2c1 - revision 10266
Production release

Copyright (c) 2007 iMatix Corporation
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Build model:Production release
Memory model: direct
Threading model: multithreaded

Compiler: gcc -c -I/u01/jeff/amqp/OpenAMQ-1.2c1/OpenAMQ-1.2c1/_install/include -O2 -Wall -pthread -D_REENTRANT -DICL_MEM_DEFAULT_DIRECT -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -DBASE_THREADSAFE -I/u01/jeff/amqp/ibase/include amq_server.c

$ ./amq_server —trace 1
OpenAMQ/1.2c1
Production release

Copyright (c) 2007 iMatix Corporation
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

2007-03-28 16:39:53: W: security warning - logins use default passwords
2007-03-28 16:39:53: I: server binding to 10.223.234.165:5672
2007-03-28 16:39:53: I: server starting in stand-alone mode
2007-03-28 16:39:54: I: server ready for incoming AMQ connections
2007-03-28 16:39:57: I: send connection.start: version_major=0 version_minor=9 server_properties=197B mechanisms=5B locales=5B
2007-03-28 16:39:57: I: recv connection.start-ok: client_properties=61B mechanism="PLAIN" response=13B locale="en_US"
2007-03-28 16:39:57: I: send connection.tune: channel_max=255 frame_max=32768 heartbeat=2
2007-03-28 16:39:57: I: recv connection.tune-ok: channel_max=5 frame_max=32767 heartbeat=2
2007-03-28 16:39:57: I: recv connection.open: virtual_host="test" capabilities="" insist=0
2007-03-28 16:39:57: I: send connection.open-ok: known_hosts=""
2007-03-28 16:39:57: I: recv channel.open: out_of_band="/development"
2007-03-28 16:39:57: I: send channel.open-ok: channel_id=4B
2007-03-28 16:39:57: I: recv queue.declare: ticket=101 queue="fqueue" passive=0 durable=1 exclusive=0 auto_delete=0 nowait=0 arguments=0B
2007-03-28 16:39:57: I: send queue.declare-ok: queue="fqueue" message_count=0 consumer_count=0
2007-03-28 16:39:57: I: recv queue.bind: ticket=101 queue="fqueue" exchange="amq.direct" routing_key="joe" nowait=0 arguments=0B
2007-03-28 16:39:57: I: send queue.bind-ok:
2007-03-28 16:39:57: I: recv basic.consume: ticket=101 queue="fqueue" consumer_tag="mine" no_local=0 no_ack=1 exclusive=0 nowait=0 arguments=0B
2007-03-28 16:39:57: I: send basic.consume-ok: consumer_tag="mine"
2007-03-28 16:39:57: W: unexpected disconnection (127.0.0.1:42257, tcl-amqp-client, 0.1, )
2007-03-28 16:39:59: I: send connection.start: version_major=0 version_minor=9 server_properties=197B mechanisms=5B locales=5B
2007-03-28 16:39:59: I: recv connection.start-ok: client_properties=61B mechanism="PLAIN" response=13B locale="en_US"
2007-03-28 16:39:59: I: send connection.tune: channel_max=255 frame_max=32768 heartbeat=2
2007-03-28 16:39:59: I: recv connection.tune-ok: channel_max=5 frame_max=32767 heartbeat=2
2007-03-28 16:39:59: I: recv connection.open: virtual_host="test" capabilities="" insist=0
2007-03-28 16:39:59: I: send connection.open-ok: known_hosts=""
2007-03-28 16:39:59: I: recv channel.open: out_of_band="/development"
2007-03-28 16:39:59: I: send channel.open-ok: channel_id=4B
2007-03-28 16:39:59: I: recv basic.publish: ticket=101 exchange="amq.direct" routing_key="joe" mandatory=1 immediate=1
2007-03-28 16:39:59: I: send basic.return: reply_code=310 reply_text="No immediate consumers for Basic message" exchange="amq.direct" routing_key="joe"
2007-03-28 16:39:59: I: recv basic.publish: ticket=101 exchange="amq.direct" routing_key="joe" mandatory=0 immediate=0
2007-03-28 16:39:59: I: recv basic.publish: ticket=101 exchange="amq.direct" routing_key="joe" mandatory=0 immediate=0
2007-03-28 16:39:59: W: unexpected disconnection (127.0.0.1:42258, tcl-amqp-client, 0.1, )
2007-03-28 16:40:00: I: send connection.start: version_major=0 version_minor=9 server_properties=197B mechanisms=5B locales=5B
2007-03-28 16:40:00: I: recv connection.start-ok: client_properties=61B mechanism="PLAIN" response=13B locale="en_US"
2007-03-28 16:40:00: I: send connection.tune: channel_max=255 frame_max=32768 heartbeat=2
2007-03-28 16:40:00: I: recv connection.tune-ok: channel_max=5 frame_max=32767 heartbeat=2
2007-03-28 16:40:00: I: recv connection.open: virtual_host="test" capabilities="" insist=0
2007-03-28 16:40:00: I: send connection.open-ok: known_hosts=""
2007-03-28 16:40:01: I: recv channel.open: out_of_band="/development"
2007-03-28 16:40:01: I: send channel.open-ok: channel_id=4B
2007-03-28 16:40:01: I: recv queue.declare: ticket=101 queue="fqueue" passive=0 durable=1 exclusive=0 auto_delete=0 nowait=0 arguments=0B
Segmentation fault

reply | options
unfold amq_server segfault by evil_ottoevil_otto, 1175125570|%e %b %Y, %H:%M %Z|agohover
fold
Re: amq_server segfault
evil_ottoevil_otto 1175130621|%e %b %Y, %H:%M %Z|agohover

This pal script achieves the crash as well.

<?xml?>
<pal script = "amq_pal_gen">
    <session>
        <queue_declare queue = "fqueue" durable="1" />
        <queue_bind queue = "fqueue" exchange = "amq.direct" routing_key="J"/>
        <basic_consume queue = "fqueue" />
    </session>
    <session>
        <basic_content size="1000" />
        <basic_publish exchange = "amq.direct" routing_key = "J"  mandatory="1" immediate="1" />
        <basic_publish exchange = "amq.direct" routing_key = "J" />
        <basic_publish exchange = "amq.direct" routing_key = "J" />
    </session>
    <session>
        <queue_declare queue = "fqueue" durable="1" />
        <queue_bind queue = "fqueue" exchange = "amq.direct" routing_key="J"/>
        <basic_consume queue = "fqueue" />
    </session>
</pal>
reply | options
unfold Re: amq_server segfault by evil_ottoevil_otto, 1175130621|%e %b %Y, %H:%M %Z|agohover
fold
Re: amq_server segfault
martin_sustrikmartin_sustrik 1175150689|%e %b %Y, %H:%M %Z|agohover

Whoa! You've managed to find race condition on connection close after we've run 24/7 tests of this kind for whole weeks. Nice. We are going to fix the bug shortly.

options
unfold Re: amq_server segfault by martin_sustrikmartin_sustrik, 1175150689|%e %b %Y, %H:%M %Z|agohover
fold
Re: amq_server segfault
martin_sustrikmartin_sustrik 1175172261|%e %b %Y, %H:%M %Z|agohover

Ok. Bug is fixed. Fix will be available in the next release.

In case you want to have a patch immediately, we've posted it to gro.qmanepo.stsil|ved-qmanepo#gro.qmanepo.stsil|ved-qmanepo.

You can subscribe to the list on: http://www.openamq.org/community.html

reply | options
unfold Re: amq_server segfault by martin_sustrikmartin_sustrik, 1175172261|%e %b %Y, %H:%M %Z|agohover
new post