Before I proceed any further with this series, I should probably mention why this work was done, and who for.

I work in a research lab at the University of Louisiana at Lafayette (ULL), in the Graduate Computer Science department (CACS), under the advisement of Dr. Arun Lakhotia.
Our primary focus when implementing this tool was to provide useful analytics to secondary users as extra annotations to the disassembly provided. The secondary focus was to experiment with in-line analysis at disassembly time.
When I mention a secondary user, I don’t just mean people. Machines perform the vast majority of assembly analysis, especially in commercial settings. We needed a disassembler that was automatic enough to produce disassembly quickly and effectively with no human interaction, but powerful enough to provide some knowable amount ofcorrect output. This information would then feed into some secondary analyzer (machine or person) who would then perform more in-depth analysis like familial clustering, function discovery, etc.
We needed one last detail. An internship I worked at for the USAF at Rome Labs, NY put me in contact with some researchers investigating instruction aliasing techniques. If you don’t know anything about x86, machine instructions are variably sized and branching instructions can target any byte-aligned address (as opposed to word or dword aligned). This means that valid execution targets can exist inside of other targets. Program authors have commonly used this “feature” in a technique called “Junk Byte Insertion,” but other uses have been documented. As it stands, few if any disassemblers handle this possibility. Ours is, to my knowledge, the first tool that retrieves and records these aliased instructions with no human interaction.
I’ll continue on with what allows this tool this unique ability in my next post.
Have some relevant links:
http://www.cacs.louisiana.edu/ (Link to our graduate department)
http://fidelio.cacs.louisiana.edu/cmps/ (Link to our undergraduate program)
http://ulsrl.org (Our website)
https://bitbucket.org/srl (Our newly minted public facing vcs)