Mon blog-notes à moi que j'ai

Blog personnel d'un sysadmin, tendance hacker

On the road to Python Protobix 1.0

python-protobix is a Python implementation of Zabbix Sender protocol. It allows you to monitor efficiently your infrastructure.

python-protobix 1.0alpha is about to be released. This release provides a lot of changes.

Let’s take a tour.

TLS support

python-protobix wasn’t designed to work over WAN. My idea on this matter is that you should use a Zabbix proxy between protobix and your server if the latest is not in the same LAN.
Zabbix Proxy will then handle the communication with the server. If needed you’ll be able to use TLS between the proxy and the server.
In my opinion, TLS in a LAN you manage brings you more problems (debugging TLS is difficult) and it only offers you a privacy you don’t really need (remember you’re supposed to control your network).

Anyway, TLS support is clearly awaited by some python-protobix users and I decided to implement it.

For now, TLS support uses Python built-in ssl module. Unfortunatly, it lacks TLS-PSK support.
I could have used other implementations like pyOpenSSL, but I don’t want to have too many dependencies for the module to work. Besides, it seems that pyOpenSSL also lacks TLS-PSK support.
Using a third-party solution has not been questionned for obvious security and privacy reasons. I found some examples of TLS-PSK support on the internet, but these haven’t mostly been updated for month and I don’t feel to endorse code inspection for such critical stuff.

Finally, Zabbix enforce TLS v1.2 protocol. While this is a clear and logic decision from a security point of view, this also means that python-protobix TLS feature won’t be available for Python <2.7.9.
python-protobix will of course work with all Python 2.7, but won’t offer TLS features for Python <2.7.9.

zabbix_sender bulk limitation

python-protobix somewhat mimics zabbix_sender behaviour. Since they provide almost same functionnalities, that’s no big deal.

Even if it’s not written in Zabbix Sender protocol specifications, zabbix_sender sends a maximum of 250 items at a time to the server.
Until now, python-protobix did not honour this limitation. I got no report of this behaviour being an issue, but it’s seems logical to respect it.

Now, python-protobix sends only 250 items at a time to the server.

Every problem has a solution

I tried to address some other issues.

Configuration options management

Problem
python-protobix can, and will, read zabbix_agentd configuration file to configure itself. You can override this configuration using command lines switches from SampleProbe.
These switches were managed with optparse which is deprecated. They also weren’t coherent with zabbix_sender options.
Solution
I migrated from optparse to argparse for the configuration switches management.
All configuration options are now stored in a specific python-protobix class: protobix.ZabbixAgentConfig.
Finally, I reworked the configuration options from SampleProbe. They’re now the same as zabbix_sender.

Code duplication

Problem
I updated python-protobix to provide new features & manage some specific use cases over time.
I tried to do it with minimum code modifications. This hasn’t been a huge success: I had to do the same change in several places to get excepted result.
Solution
Many part of python-protobix have been rewritten and thus simplified. The main consequence is that I changed the protobix.DataContainer API without backward compatibility.
This is why I intend to release python-protobix as 1.0 and not 0.2 or whatever. The good news is that migration path should be very easy, especially for protobix.SampleProbe users.

More tests

Problem
I’ve had one memory leak issue with python-protobix. Nothing scary, but playing with sockets without paying enough attention can be harmfull, and indeed was. Of course it’s easier to spot with a test suite but, at this time, I hadn’t covered that sort of tests.
Solution
While working on python-protobix 1.0, I tried to write tests before writing code. I can’t say it was exactly “Test driven development”, but it was still quite interesting to do so.
Since I had to implement TLS, and so play with sockets, I wrote a test simulating a long running process and controling its memory usage. I should now be able to detect a memory leak before release. I also tried to cover as many code as I could. I had good results, even if I still have to learn how to efficiently mock things.

More documentation

Documentation wasn’t up-to-date. This will be fixed and, for future releases, I’ll try to write documentation before writing any line of code.

Conclusion

With many new features to come, including long awaited TLS support, python-protobix should be released shortly after the Zabbix Conference 2016 in Riga.
New features and usages examples will be detailled in separate blog entries.