DPythOS - Distributed Python OS for Host Management.

Posted 14 May 2002 at 04:22 UTC by lkcl Share This

DPythOS provides the means to monitor and manage network hosts and services.

Unlike a lot of monitoring packages, DPythOS provides administrators with a way to _correct_ problems if they occur, and to notify them if that also fails.

Download:

http://sf.net/projects/dpythos

Details:

DPythOS scripts are run in a restricted and minimally extended python environment. Scripts can be run or scheduled from the command-line, and also scheduled from a [very basic!] HTML interface.

Scheduled scripts are run priority-based on a per-host basis, with the maximum number of simultaneous scripts being run by the monitor daemon (monitord.py) limited in a config file option.

DPythOS runs [at present] a telnet, or ssh, or HTTP connection to a remote host. Data can be sent down the connection, and the results received and acted on.

Implementation Fun And Games:

people really interested in what kind of weird uhh... stuff... can be done with Python will be intrigued to know that DPythOS does the following:

- uses the newly introduced BaseServer in SocketServer.py to create a SQL database server "thing".

basically, it locks, polls entries in the script scheduler table, reads the entry (which includes the script name, host name and priority), actions the script, removes the entry, unlocks.

this was the whole reason why i submitted the BaseServer patches, but couldn't release the code that _used_ it, at the time.

- overloads ihooks such that "import" first checks whether the script being imported is in the SQL database (!!! :) and provides the functions from the scripts table instead of off of the filesystem. [cool :)]

in this way, common libraries can be developed and managed from the central database. for example, the "loginlib" SQL script loads the login username and password for the target host, and actions the default login method (telnet or ssh) in order to provide a command-prompt by which the host may be managed.

all this is done under database control. cool, huh?

- overloads rlcompleter such that in script interactive mode (suitable for use as a superuser login shell, amongst other things) host names, script names and DPythOS functions have readline tab completion available.

teehee.

- uses a TelnetBase class (which wasn't suitable for release in Python2) to provide Popen3Telnet (via which ssh is then accessed), HTTPTelnet, and anything else you fancy to add.

basically, the write and the read_until, read_some functionality of Telnet was _way_ too good to miss up on for ssh, http and popen3 wrapping.

so i didn't [miss it, that is].

- Database abstraction. i wrote a SQL database abstraction system that constructs SQL statements in sections. if you hate views, or if your SQL server doesn't support it, the abstraction layer is a good way to construct complex SQL rules and joins without duplicating loads of text.

also, bundled with the distro is pyxsqmsll2000, which is a stupid name for a Microsoft SQL 2000 XML access client. SQL 2000 has an XML interface which you can enable: pyxsqmsll2000 can send SQL statements via HTTP and actually understands the results that come back.

yippee.

- python's restricted environment is used, with some imports pre-added for its use. in order to transparently provide the restricted environment with DPythOS features, all of the functions have to be added in from a couple of base classes.

it's a little weird, it also allows the help functionality to be auto-generated, and you should really take a look at the code to see what i mean, it's a little bi tricky to explain. DPythOS/script.py - class DpythosDelegate.

BUGS:

- all versions of ssh are non-interactive with respect to password input. i had to patch open-ssh to accept passwords on the command-line (ssh password:user@host).

- last time i checked, there were locking issues with the version of MySQL i was using (2001). the same table in one of the databases is opened _three_ times under different names, one with write access and the other two aliases with read access.


Sounds like you could use Twisted., posted 14 May 2002 at 09:06 UTC by spiv » (Master)

If you're doing that much stuff with networking in Python, you could almost certainly benefit from Twisted Python. It'd make a much better base for implementing HTTP and telnet stuff than doing it yourself.

I'm wondering what you can actually do with this system you describe... You basically spend the whole article breathlessly describing bits of the framework which you think are cool :)

Hmm. The project page is a bit out of date... you're probably better off going to www.twistedmatrix.com.

erm, you patched ssh?, posted 14 May 2002 at 18:35 UTC by washort » (Journeyer)

why are you using password authentication? for automated connections, RSA auth seems like a much saner approach, and removes the need for patches.

twisted & ssh, posted 14 May 2002 at 18:55 UTC by lkcl » (Master)

hm. regarding twisted: i think you're probably right! if i had more time on my hands, i'd do a more in-depth investigation. or if rewriting a 2nd version. DPythOS was 6 months of development - which i don't entirely fancy duplicating, and certainly can't at the moment, hence its release as open source.

regarding ssh: patching ssh was the simplest way to make ssh act in the same way as telnet. in the very basis usage of popen3 to wrap ssh, the loginssh script doesn't even check for "you are connecting for first time, check fingerprint etc etc.".

reading back the text from ssh via popen3 and searching for "check key fingerprint are you sure [yes/no]?" text is very easy: i just didn't have time to add it.

adding ssh at _all_ was proof-of-concept, as telnet was, at the time, sufficient. badly insecure, but sufficient.

answer: adding better ssh usage is easy to do. this _is_ version 0.0 after all :)

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page