This week I learned about some esoteric looking (but simple to solve) errors.
RabbitMQ and Chef
I was assisting a client with setting up and testing a new Chef 0.10 instance. He completed the steps we had outlined an used in previous environments, but we were getting a strange error when attempting to run "knife configure -i".
Creating initial API user... ERROR: Server returned error for http://localhost:4000/clients/root, retrying 1/5 in 3s ERROR: Server returned error for http://localhost:4000/clients/root, retrying 2/5 in 6s ERROR: Server returned error for http://localhost:4000/clients/root, retrying 3/5 in 11s
We tried several things, checked some logs when we found this line with a substantial stack trace:
merb : chef-server (api) : worker (port 4000) ~ Connection failed - user: chef - (Bunny::ProtocolError)
After some head scratching and iterating on search terms I found a blog post by Jonathan Polansky indicating this was a problem with RabbitMQ not having the Chef specific configuration. He helpfully provided the commands to run that only needed tweaking to add the default password for the RabbitMQ user 'chef' (which is, BTW, 'testing').
# rabbitmqctl add_vhost /chef # rabbitmqctl add_user chef testing # rabbitmqctl set_permissions -p /chef chef ".*" ".*" ".*"
Thanks to Jonathan for finding that!
YUM and Proxies
Working with the same customer on spinning up a test machine with their new Chef infrastructure we came across this error:
Setting up repositories http://[REDACTED]/RedHat/repodata/repomd.xml: [Errno 4] IOError: <urlopen error (111, 'Connection refused')>
We could retrieve the file via CURL from the same machine without a problem but yum refused. Finally we figured out that our standard configuration in Chef for yum required a proxy. We had set this up in other environments, but we have not retrofit the new Chef instance to be configured by Chef.
Source: chef-server connection failed to rabbitmq using bunny protocol as user chef (http://jonathanpolansky.com/2010/10/chef-server-connection-failed-to-rabbitmq-using-bunny-protocol-as-user-chef/) by Jonathan Polansky
