This is from experience of having done network-reverse-engineering of Samba TNG for four years. The bias
is therefore specifically geared towards dealing with Microsoft, but it
applies to all big companies.
0) ASK FOR THE SPECS! i know this sounds weird, but it's a very
important step. most of the reverse-engineering laws (esp. the EU ones)
require that you be able to PROVE that 'the information was not
available by any other means'. and only THEN can you do
so, you must ask. send an email to Mr Bill Gates. do a follow-up
telephone call [no, i'm actually not joking, although it does sound like
a funny thing to do: hi, mr gates, please give us all of your secrets we
want to do as well as you]. send out letters by courier to 1 microsoft
way, by email to public forums where the people developing the software
are known to hang out / respond.
wait for six weeks. if you get no response, assume that [GET LEGAL
ADVICE!] you've done enough, and you can proceed.
1) Buy licenses for all - ALL software - you use. Companies have been
successfully sued not for reverse-engineering but for not having
licenses for the software (or in some cases hardware) they were
2) Find out what the legal attitude towards reverse-engineering is in
the country you intend to work in. if it's not legal there, find some
employees in another country, and find out what the legal aspects are
w.r.t. importing that software.
3) Find out the legality of, and follow this:
a) employ one person to do the reverse-engineering, to Hack Up Some
b) employ a second person who can talk to a) who acts as a firewall,
generating a specification which is published PUBLICLY and then give it
c) employ a third person who has never heard of, met, seen, breathed in
the same room as a), to implement b's spec.
repeat until c completes the implementation. at least one large company
reversed-engineered the specification for and a clean-room
implementation of X-Windows this way, where ab and c were teams not
4) follow the interoperability / rev-eng guidelines. in the EU, as
mentioned above, the laws are that you can only rev-eng for
interoperability only, where info is not available by any other means [i
assume that this excludes microsoft's NT source code license - available
at $1m per year - because it is a read-only license...]
in the U.S. okay. the u.s. is a bitch. i cannot BELIEVE how the DCMA
xxxx got through. anyway, that's been shot to pieces, discussed to
death etc etc and of course, they have more money than us, so they win
[oo sarky today, aren't i?]
... however: read it carefully [the DCMA]. you will note that there are
some exceptions. namely, security companies and anti-virus companies.
5) go work for, set up, or create, a security company. as part of the
research of that company, do the following:
a) produce security reviews and audits of the program being
b) BE RESPONSIBLE. THIS IS IMPORTANT, LISTEN UP. get onto the GOOD
side of the company whose product you are reverse-engineering. in
microsoft's case, that's the email@example.com people. DO NOT
UNDERESTIMATE HOW IMPORTANT THIS IS.
send them reports, repro code, DO NOT BLAB TO THE PRESS. follow the
guidelines for bug-fixing / security-bug-fixing, namely:
- report to vendor - *full* disclosure including repro code. do NOT
blab to public.
- send follow-up 2 weeks later if no response, saying, 'if no response,
we must go public to get your attention, in 6 weeks time'.
- in 8 weeks time, make a RESPONSIBLE report not outlining how to create
a virus or a public death-sentence program. you don't want to be known
as the people who helped destroy the internet....
if at ANY time you hear from the vendor:
- do not blab to the public. work WITH them to produce a security
bulletin, to do a simultaneous announcement of your report and theirs.
GIVE THEM TIME TO FIX THE PROBLEM RESPONSIBLY AND PROPERLY. you don't
want them to get irritated at having to pull out all the stops, come out
with another security-fix later 'cos you didn't give them time enough,
it will only bite you later [see Last Resort, below].
c) produce a full specification (this can be your 3a, above: and heck,
you even get the good-will of the vendor, too!) ... AND PUBLISH IT. do
it as a book (e.g ISBN 1578701503 - DCE/RPC over SMB: Samba and Windows
NT Domain Internals, shameless-plug, shameless-plug).
6) get life insurance. oh, you think i'm kidding?? then why did four
separate individuals advise me to be ever so, ever so careful, when i
started in august 97 on the nt domains protocol?. why have three
separate individuals sent me parts of their work asking that it not be
disclosed that they wrote it? because they were afraid of the 800lb
gorilla and what it might do to them, their careers etc. think about
it. you're taking on a multi-million dollar company who can't _buy_ you
- you're open source, for pity's sake. their alternatives involve
patents and legal recourse (last resort: it would be _really_ bad
press), employing the developers, or getting rid of them by other means.
7) WRT crypto:
a) if it's passwords / authentication, then [GET LEGAL ADVICE] it's
acceptable to produce NON-GENERIC code that does not have a publicly
accessible interface to be able to encrypt data.
b) send a copy of your code to the U.S. Dept responsible for exports.
apparently, this is now sufficient [GET LEGAL ADVICE!] to export the
code from the U.S.
c) if you have non-U.S. employees, GET A BXPA LICENSE. the contents of
your non-U.S. employees heads are then 'countries-in-their-own-right',
to which you can 'export' your valuable, and of course, U.S. developed,
military-grade munitions. these employees then become subject to the
same laws as U.S. citizens. they may talk to other U.S. citizens but
not to anyone else.
this gets you out of the ridiculous situation where your non-U.S.
employees can email people in the company, but they can't respond.
why? because _you_, the non-U.S. employee, have developed some munition
that is imported to the U.S. [no laws broken here, yet]. but, if those
employees respond, they are EXPORTING those munitions [to your head,
from whence it came...] and THAT is illegal. so don't do it.
either that, or just post absolutely bxxxxy everything onto public
forums. DO NOT engage in private conversations with individuals
off-list about crypto. then it's covered [GET LEGAL ADVICE] by free
speech laws... hmm....
8) have all intellectual property owned by an off-shore company [this is
not legal in the U.S].
so, you got onto their good side. you helped them improve the quality
of their code, by helping them fix important bugs. you develop a good
relationship with some of the developers, even: and they let you in on a
few secrets - that some of them actually *support* what you are doing,
because of the massive benefits to *their* product. ... and, get this:
you know how lonely it gets, working on this stuff? it's so advanced,
there are very few people out there who understand it. well, guess
what? the people who originally developed it feel the same way. and
despite the confidentiality clauses, actually having someone to talk to
who understands and respects what you have achieved (working out what
they did _and_ doing a good job), even if they _are_ a little scared of
you :) is worth a thousand confidentiality clauses. a breath of fresh
air is always welcome [except if it's bitterly cold air... ]
The Last Resort
... then, the Marketing Machine got Mean, decided to Take You Down. The
Legal Machine Gets Involved. you end up Underground. well, if that
happens, you know enough to produce some really nasty viruses. carrot
didn't work, Big Stick Time...
darn, i almost forgot network reverse engineering.
network reverse engineering is [GET LEGAL ADVICE] legal. why? because
it's publicly accessible data. there's nothing to stop you duplicating
or parroting data you can see on a network [yet. GET LEGAL ADVICE].
so what approach should be taken to do network-rev-eng? well, this is
important: develop a client and a server at the same time.
why? because you can use one to bootstrap the other. it goes like
this. Real means the one you wish to network-rev-eng; stub means the
one you are developing:
- make a connection attempt from a Real Client to your Stub server.
record the request into a file.
- plug the recorded request into your Stub Client. replay it to your
Real server. the Real server will respond [hopefully! this isn't
actually guaranteed, see below!] to the Stub Client. record the
response into a file.
- make the same connection attempt from a Real Client to your Stub
server. when your Stub server receives the request, playback the
recorded response. [hopefully!] the Real Client will buy that and it
will move to the *second* request, and repeat from above.
- you need to examine the data being recorded. network traces, which
show really obvious things like names of servers, names of clients,
names of users, and strings in general, you soon get used to 'seeing'
those, and it's pretty obvious, *duur* well if there's a client name in
the request, *duur* i have to send *my* client name back.
so, to help you do this: make sure that Real client and Stub client, and
Real server and Stub server have names *of the same length* :) :) you
can then in-line search/replace the strings of the right names :) :)
w.r.t. strings, look especially for string-lengths. w.r.t
variable-data-blocks, look for data-lengths. you *can't* do network
requests in structure-packing *without* having data / string lengths
(maybe strings you can null-terminate) except over UDP of course. and
of course except having escape-sequences like you do for xml, html,
etc., you know the drill.
- checksums, digital signatures and crypto in general tend to screw up
the above process. at that point, you need to follow advice laid down
in this article [GET DAMN LEGAL ADVICE]
when you have encrypted data, then assuming that you have managed to
crack the ecryption, the above process becomes REALLY important: staring
at network traces is useless.
- stare at network traces. i mean, stare. make several. compare
them. make more doing the same thing, compare them. stare at them.
network traces are your friend. it _will_ sink in - eventually. like,
after two months.
i use vi at the console, so i have two text-dumps (one on each window)
and i flick backwards and forwards, then page-down, alt-f1, alt-f2,
alt-f1, alt-f2. page down alt-f1 page down alt-f2 alt-f1 alt-t2.
differences show up *immediately* - remember, eyes are good at detecting
*changes*. make it easy to detect changes.
if using Netmon, use Netmon v1 because netmon v2's display capabilities
are xxxxed up [it uses internet destroyer to do the displaying]. if you
do Alt-W[indow] 1, Alt-W[indow] 2, repeat, you find that the two windows
slowly migrate one pixel down at a time!!!! _not_ very helpful :)
- make only one small change in the Client / Server setup and compare
the sniff results. e.g. change the name of the client (or more
specifically, change the _length_ of the name of the client).
- develop an automated decoder tool. plug the above-recorded requests
and responses into it.
pleeeeaaase develop the automated decoder tool into a real IDL
compiler. TNG has 50,000 lines of *HAND GENERATED* structure-packing /
unpacking code. it's very well structured code, but.... never mind, i
just don't recommend anyone doing it, that's all :)
get specs where you can on the wire-format. write or find an IDL
compiler. please. you will feel a lot better for doing so :)
anyway, if i think of anything more, i'll post it here, later.