nova-agent not starting with import site’ failed

It’s amazing how long it’s been since I last posted here! I’ve moved to a new role more on the Infrastructure side of things and have been busy.

Today I wanted to share how to correct a bug that nova-agent version 0.0.1.38 has.

Problem

When starting it you would see this

# service nova-agent restart
'import site' failed; use -v for traceback

Unhelpful, right?

Investigation

Looking at the log shows that the problem comes from the xscomm.py script:

# tail /var/log/nova-agent.log
2014-05-06 17:57:17,299 [ERROR] [EXC] InvalidArgError: Invalid argument specified
2014-05-06 17:57:17,299 [ERROR] failed to parse config file '/usr/share/nova-agent/nova-agent.py'
2014-05-06 17:59:50,892 [ERROR] Failed to run python code: A python exception has occurred:
2014-05-06 17:59:50,892 [ERROR] [EXC] Traceback (most recent call last):
2014-05-06 17:59:50,892 [ERROR] [EXC]   File "/usr/share/nova-agent/nova-agent.py", line 41, in <module>
2014-05-06 17:59:50,892 [ERROR] [EXC]     xs = plugins.XSComm()
2014-05-06 17:59:50,892 [ERROR] [EXC]   File "xscomm.py", line 44, in __init__
2014-05-06 17:59:50,892 [ERROR] [EXC] InvalidArgError: Invalid argument specified
2014-05-06 17:59:50,893 [ERROR] failed to parse config file '/usr/share/nova-agent/nova-agent.py'

A quick find will show that that script is a compiled python script here:

# ls -lah /usr/share/nova-agent/0.0.1.38/plugins
total 28K
drwx------ 2 root root 4.0K May  6 18:05 .
drwx------ 7 root root 4.0K May  6 18:01 ..
-rwx------ 1 root root  253 Nov  7 18:17 __init__.pyc
-rwx------ 1 root root 3.1K Nov  7 18:17 jsonparser.pyc
-rw-r--r-- 1 root root 3.5K May  6 18:05 xscomm.pyc

Hmm… compiled eh?

Solution

Ok, let’s go to the source on github. Once there you can see that xscomm.py has been updated 3 months ago because of some startup failures. Looks promising, so let’s go ahead and get it:

# cd /usr/share/nova-agent/0.0.1.38/plugins

# wget 
--2014-05-06 18:04:41--  
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.31.19.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.31.19.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5709 (5.6K) [text/plain]
Saving to: `xscomm.py'

100%[==============================================================>] 5,709       --.-K/s   in 0s

2014-05-06 18:04:41 (103 MB/s) - `xscomm.py' saved [5709/5709]

Move the old one out of the way:

# mv xscomm.pyc /tmp

Aannnd start!

# /etc/init.d/nova-agent restart && tail -1 /var/log/nova-agent.log

2014-05-06 18:05:09,903 [INFO] Agent 0.0.1.38 started

I hope this helps.