Ruby and DNS.

While i reviewed the source-code of a FakeDNS-utility i wrote a while ago, i saw the need to make this tool a bit more ruby liek and a lot of improvement could be done, especially regarding the handling of dns-messages.There are a few different gems for handling these messages, including a standard library for Ruby. I decided to check out the Resolv-library to handle the DNS-packet it self, but quickly discovered that this library wasn't very well documented at all.

When i browsed through the source-code in 'ruby-src/lib/resolv.rb' i realized that the code had some nice objects which was flagged with # :nodoc. While reading source-code is great fun during the weekends and all that, one could of course uncomment all the "# :nodoc" tags, and do a create an rdoc on that particular file to get the docs containing all the goodies....

This is a short post on the very basics of Resolv and how to build a DNS request / response ready to be used as a server/client scenario.

Message

While going through the source-code i discovered Message, a class to handle a DNS-message, which even comes with a class-method called #decode(data). This particular method takes UDP-data as argument and populates a DNS Message-object based on the data given. This makes it pretty easy to create DNS-messages indeed.

Instance objects.

There are several ways to manipulate and work with an message-object it self(eg after decoding some data). Message-object it self have a few methods which makes it pretty easy to extract and add information into each of the sections of the DNS-message. A few objects are needed before any of these methods are used. For instance in a query a name and a typeclass is required to add a question into its section.

These classes can be created with Resolv::DNS::Name.create(string), and Resolv::DNS::Resource::IN::A would be an example of typeclasses. To add questions and answers into the message-object the following methods makes it quite easy to work with message-objects.

add_question(name, typeclass)
add_answer(name, ttl, data)
add_additional(name, ttl, data)

add_authority(name, ttl, data)

Typeclass can be one of the following as shown in the picture below.

Typeclasses defined in resolv.rb.

While typeclasses are constants they can also be used to create new Resource object. By using Resolv::DNS::Resource::IN::A.new we can pass this on to a message question / answer section. For my purpose i only need two of the methods mentioned above to create a DNS-response, but one could also add additionals and authority the same way as add_question-method.

Query

To create a dns-query for www.google.com following method is used:

# Create a new DNS message.
request = Resolv::DNS::Message.new

# Add a random 16bit unsigned integer to message' ID.
request.id = rand(0xffff)

# Add a new Name-object into message' question section.
request.add_question(Resolv::DNS::Name.create("www.google.com"), Resolv::DNS::Resource::IN::A)

Response

Based on the request-object i can easily create an response as shown below.

response = request
# Question / Response (1) flag and set recursion available flag if recursion desired flag is set
response.qr = 1
response.ra = 1 if request.rd.eql? 1

# Grab domain name. 
domain_name = response.question.first.first
# Grab typeclass
resource = response.question.first[1]

# Create a new typeclass object based on the resolved IP-address.  
resource = resource.new(Resolv.getaddress(domain_name.to_s))

# Add domain_name-object, a random 16-bits unsigned integer as ttl and resource-object.
response.add_answer(domain_name, rand(0xffff), resource)

Use Message.decode to parse incoming data instead of creating a new Query-message, put this together with your UDPServer and suddenly you have created a simple DNS server using Resolv.getaddress.... Surely this is a lot easier than first represented in FakeDNS.

Software hooking

Introduction

I have every now and then encountered malware which steals or manipulates data from other software ran in a user environment. In "Practical Malware Analysis - A hand-on guide to dissect software" they wrote a couple of lines about a topic called "hooking". In this book they discuss two different techniques which often are used in Windows user-mode rootkits, namely IAT Hooking and Inling hooking. Recently i found an article discussing software interposition, which is another word for hooking software-functions, and found it quite interesting to read that within Linux one could pretty easy intercept symbols which other software depend on.

There are various ways to perform these kinds of software hooks, and i will in this blog-post discuss hooking in a Linux environment rather than in a Windows environment. While these kind of techniques are often implemented into malware, one could also use it to debug software, create statistics, look for memory leaks as well as during malware analysis.

Shared Library

In computer software developers rely on using dynamic linked libraries which is a set of functions one include into your own code. C programming language has a set of standard libraries which are defined in a file called libc.so.6. Functions like printf(), rand() and so on are defined there for our pleasure to use, and are shared amongst different software which are relying on these functions to perform the wanted actions.

When source-code is compiled, the compiler link the dependencies between your program and the library through a process called linking. The linker will take the object-files created by compiler and combine them into one single executable, and are thus responsible for placing the references to the wanted symbols. Then it will place the reference into a certain section inside the binary, which is specified by operating systems and again file-format (In this case: ELF). At runtime the symbols are resolved by the first library that provides the functions used by the program.

When compiling a shared library it is important to use -shared and -fPIC compile options since we are building a shared-library and shared-libraries should  be Position-Independent Code. By using PIC we ensure that the library can be mapped into any memory address without being modified, and executables can map this address into its own address space and call it properly without any problems, regardless of the library's address.

Linking and compiling are often mentioned as one and same thing as this is happening during compile-time, but are quite different.

For a better understanding of libraries, static libraries and shared libraries see Dynamic Linking in Linux and Windows.

Function interposition

Function interposition, often referred to as hooking, is a technique to intercept dynamic library functions by writing a wrapper-library which replaces the actual wanted symbol. It can be used by developers to debug their program, run all sorts of run-time statistics and so on. It is very often implemented into malicious software, and it can be used to analyse malicious software in a different manner than i am used to.

As mentioned above, the symbols are resolved by the first library, so if we could get our wrapper in between the software and standard C library, we should be able to intercept the symbols used in a program. So, how do one perform this kind of magic?

First of, in Linux there are LD_PRELOAD environment variable, which could be very easily described as dynamic-linker-preload. This will tell the loader to load library stated in this variable before standard libraries. Information regarding this variable is found in the "GNU linker"-manual (man ld). At this point i raised the security alert since this is allowed in Linux this easily; but this environment variable has some limitations; We can for example not intercept software which requires root-access (setgid, setuid-flags set).

With all this in mind i could easily write a simple library which replaces the rand()-function - it is an easy first choice (examples are to be found HERE).
The function rand() is defined in stdlib.h as such: "int rand(void);". The replacement could look something liek this:

#include <stdlib.h>
/*
* Compile:
*  gcc -shared -fPIC -o rand.so 02-rand_replace.c
* Run:
*  LD_PRELOAD=$PWD/rand.so ./randy
*/
int rand(void) {
    return 1; // fixed rand
}

 
But this isn't very sophisticated at all as we have no way to get back the real value of rand, this could clearly create errors during runtime if the software rely on a certain symbol, as they most of the times does ;)

C Dynamic Linking API

Fortunately there is a C-library for dynamic linking which makes it possible for us to work with dynamic libraries. By reading the documentation i found out that by creating a wrapper and to use dlsym-function i could fetch the old instruction. The documentation states the following:

The RTLD_NEXT flag is useful to navigate an intentionally created hierarchy of multiply-defined symbols created through interposition. For example, if a program wished to create an implementation of malloc() that embedded some statistics gathering about memory allocations, such an implementation could use the real malloc() definition to perform the memory allocation-and itself only embed the necessary logic to implement the statistics gathering function.

So by using dlsym with RTLD_NEXT-flag and the function to hook, we could perform our wanted function-interposition, do what ever with the data and return the "old_result" if thats desirable. In the old rand()-function i then created a point to int old_rand, to hold the results of the symbol actually called, and then we can place dlsym into that pointer. Because i use the RTLD_NEXT flag i need to either compile with -D_GNU_SOURCE or define it inside the source-code, as the code below.

 #define _GNU_SOURCE
#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>
/*
* Compile:
*  gcc 03-hooking.c -o hook.so -shared -fPIC -ldl
* Run:
*  LD_PRELOAD=$PWD/hook.so ./randy
*/

int rand(void) {
    int (*old_rand)(void);
    int res; 

    old_rand = dlsym(RTLD_NEXT, "rand"); // Hook first/next rand()
    printf("Hooked on rand()\nBad boying starts here...\n\n\n");

    res = old_rand();
    return res;
}

By using the program above i could easily interpose as a rand() function and perform wanted actions where the "badboying" starts. I decided to call it badboying as i was relating this to earlier malware-analysis, but one could perform all sorts of actions inbetween these lines of codes.

 Okey, how fun wasn't that? Well, the examples was not fun - but this technique seems quite powerful, and my immediately thought was to perform these kind of actions on a malware-sample - although i would believe it is possible to detect this kind of hooking and avoid it completely. Sounds liek a new blog-topic.

References

http://www.symantec.com/connect/articles/dynamic-linking-linux-and-windows-part-one

 http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html

http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/

http://www.jayconrod.com/posts/23/tutorial-function-interposition-in-linux

http://tldp.org/HOWTO/Program-Library-HOWTO/dl-libraries.html

Scraping with Ruby

Lately i have been fiddling around with several libraries which simplifies the whole process of working with HTML pages in Ruby. For the simplest of HTML-handling "open-uri", a default library that comes with Ruby, should be sufficient. To handle html-code in a more complex way i have tried both Mechanize and Nokogiri to help me extract wanted information based on certain criteria i have. The process of extracting and parsing data obtained from web-sites are commonly called "scraping". There have been written numerous articles and blogs on the subject, so for the sake of it - here is yet another one:)

As an example i am going to extract electronic components from a vendor, where i want specific information of each component - information like Serial-number, Name and price are all relevant. I want to place the components in its category since i am going to fetch several different component-types, such as resistors, transistors, diodes, IC, Crystals and resonators.

Scraping

If i were to extract all this informasjon by hand it would be quite tedious work, so what isnt more fun than to create a script that simulates humans browsing through wanted paged and extracts all this informasjon for us, which again we can use to manipulate on a later time by storing into files. The way we search / scrape for relevant information is by searching for certain elements in the source-code of each page by using so called CSS-selectors. As we might know, CSS (Cascade Styling Sheets) are used to keep page styles, fonts and such separate and easily available from your code. In CSS we often rely on using selectors to mark up which part of the code each style should be declared on. By using these selectors while parsing we can extract wanted information based on which element we want the selector to grab.

As a side note i want to mention that when ever one build scraping-tools and web-crawlers we should always respect the sites "robots.txt" found in the root of web-domains (http://www.example.com/robots.txt). This file commonly tells robots how they should read a site and where they have and dont have access, read more here - Robots exclusion standard. Although we have pretty decent control over our bot and where to go so it is not a subject in this matter. Though i decided to check the robots.txt-file; They were only rejecting access to the "cgi-bin" folder.

Ruby gem

In ruby we have several gems which does the whole web-scraping process pretty easy. From simple web-scraping to more advanced html-handling liek filling out forms and to process cookies etc. In this example i do not need more than simple CSS-selectors, so i will use Nokogiri to assist me. Nokogiris authors describes it like this:

Nokogiri (鋸) is an HTML, XML, SAX, and Reader parser. Among Nokogiri’s many features is the ability to search documents via XPath or CSS3 selectors.

Okey, this is good and all that - now we need to take a look at the page and see what information we can start with.

Scrape teh page

Lets move to the main site belonging to the vendor and see what it looks like. When we go to http://www.ehobby.no the main page will show some kind of greetings, at the left we can see categories-submenu where we find the following text "Komponenter" (Components) this is a good starting point, now we need to find the tag used to display this text.

Finding elements is pretty easy when using Firebug-extention which allow us to "inpect" certain elements on a site and display the requested element in Firebug's own analysis-window as shown below.

Since this is all the information we need from the main page it surely would be alot easier if we just gave that URL to our bot instead of writing a small procedure to extract this portion of the code. Lets move on to the category-listings page where we will find our wanted categories of each component. Now we can start looking for the wanted elements and put them right into Nokogiri as CSS-selectors and extract each category. To hold only the wanted categories i created a hash with key-names of each of the wanted categories, each key holds an array as value to store each component inside.

Let us try to use Nokogiri to show show us the url to the components-type we want. The elements we can use are shown in picture below where we will see how each category is wrapped inside a div-tag with class-name "categoryListBoxContents".

By using IRB for testing purposes we don't hammer the web-site with requests as we create a Nokogiri-object of the page and search for the elements by using CSS-selector shown above. A picture of how the test-script looks like is shown below.

This will output Category-name and URL to components in each category. It is quite easy to extract the information this way as we just have to identify the elements and place those into Nokogiri, but when we do look at the resistors components-list (for example) we will see it shows only 10 results pr page, "Showing N of N (out of N products)", so we need to create a method finding out how many results we have obtained and how many pages there actually are. Firebug help us find the selector to identify these number and extract them. Below is a picture of how this info is extracted and used in a method in ruby.

All we need now is to extract each component and put it into the right hash-key of categories. To scrape the components i could use the css-selector "table .tabTable tr", but this gave me 11 results pr page - so it did actually include the text of each column on the page as well as all the 10 components. To drop out the column-text i had to look for <td>-tag inside the selector shown above, if this contained 4 elements we assume that is the column we want and place them into an array. The only problem is, because i use a very nasty split to extract the prices we dont get the right price - since i only extract one decimal (and split them at ","). But for this example i guess this is all good.

At this point we can handle the extracted data just as we want - we already have them in a container. But as the complete script will reveal, i wrote them into a simple text-file. Now all we need is a complete scrape_index-method, which are going to loop through all the categories, fill them with wanted information.

Now we have completed the scraper to extract wanted information and store it in a simple way.

This is the basics of web-scraping, script can be found on me drive. 

Flere NSM utfordringer

Sist post omhandlet ene og alene om utfording 6 (shellcode analysis) som Nasjonal Sikkerhetsmyndighet har gitt ut i sin blog  i et forsøk på å bemanne en rekke stillinger. . I denne posten analyserer jeg flere av de andre utfordringene jeg klarte. Utfordringene er tilgjengelig her.

Utfordring 1

Kryptert tekst-streng:

PNRFNE PVCURE RE SBE YRGG.
SVAA SYRER HGSBEQEVATRE CNN FVXXREURGFOYBTTRA GVY AFZ.

Den krypterte teksten er Cæsar cipher, eller nærmere bestemt ROT13 kryptering hvor man endrer bokstavers basis-plass i alfabetet for å kryptere tekst, f.eks som å forskyve alfabetet 13 plasser til høyre som det gjøres spesifikt i ROT13 cipher.... Ved hjelp av Ruby og String.tr-funksjonen kan vi gjøre dette på en effektiv måte.

class Rot13
 def self.decode(encoded_string)
  encoded_string.tr("A-Za-z", "N-ZA-Mn-za-m")
 end
end

encoded_string = "PNRFNE PVCURE RE SBE YRGG.\nSVAA SYRER HGSBEQEVATRE CNN FVXXREURGFOYBTTRA GVY AFZ."

print Rot13.decode(encoded_string)

#
# Output:
#     $ ruby 1.rb 
# CAESAR CIPHER ER FOR LETT.
# FINN FLERE UTFORDRINGER PAA SIKKERHETSBLOGGEN TIL NSM.

Caesar cipher Ruby-gem

Utfordring 2

ULFC://TXBYS.AKY.FLMG.FA/GSD/QM/GGXAEVDVFSRF? 

I oppgaven ser vi en tekst-streng som ligner mistenkelig på en web-adresse. Siden den kun krypterer bokstaver velger jeg å prøve ut Vigenère cipher-tabellen og. Siden vi ikke har peiling på nøkkel-ordet som eventuelt skulle brukes i cipher kan vi først prøve å se for oss at det står "http://blogg.nsm.stat.no/" om ikke annet, så finner ihvertfall deler av nøkkelen ved å konvertere http. Det viser seg å være NSM som er nøkkel-ordet brukt til å kryptere web-adressen. Dette har jeg prøvd å illustrere i bildet under, hvor på toppen man ser nøkkel ordet, og i kolonnen til venstre ser vi dekrypterte dataene.

Vi ser at ULFC blir NSMN så da har vi nøkkel-ordet og kan fortsette kryptering, enten ved hjelp av ruby eller på papiret. Uansett vei så ender vi opp med dette når strengen er dekodet: http://blogg.nsm.stat.no/tar/du/utfordringen?

Vigenére cipher Ruby-gem

Utfordring 4

Bildet som vises frem inneholder steganografisk data og det finnes mange forskjellige metoder for å skjule informasjon  ved hjelp av steganografi. Steganografi er en måte å skjule informasjon ved at meldingen opptrer som noe helt annet, som f.eks bildet. Første hintet får man når man søker etter fil-informasjon, da ser vi en kommentar på bildet som er uleselig. Kommentaren er krypert og når jeg ser på den krypterte tekst-strengen får jeg lyst å prøve ut Base64-biblioteket i Ruby / IRB.

Kommentaren er kryptert ved hjelp av Base64 kryptering og da kan vi jo egentlig bare tippe en gang på hvilket passord man kan trenge for å ta ut dataen som ligger skjult i bildet ;) For å hente ut dataen som ligger i bildet bruker jeg et program som heter "steghide", og vi prøver med NSM som passord. Under ser vi resultatet.

Shellcode - NSM utfordring 6

Nasjonal Sikkerhetsmyndighet (NSM) har lagt ut en rekke utfordringer / oppgaver som mann kan løse. Jeg satte meg ned i helgen for lese litt på oppgavene, etter å ha lest gjennom de en gang løste jeg noen av, andre var igjen umulig for meg å løse. Denne blogg-posten omhandler "utfordring 6" hvor mann skal analysere noe som ser ut som en shellcode.

Koden vi ser er skrevet som ren maskin-kode (assembly op-codes). Slik maskin-kode kalles ofte for shellkode og blir brukt i forbindelse med utnyttelse av sårbarheter i programmer og OS. Man ønsker at shellcode skal være så liten som mulig, og er ofte bare små program-snutter, slik som vi ser i eksempel under. Navnet kommer av at man ønsker å starte ett skall/shell hvor angriperen har full tilgang til kommandoer etter at en sårbarhet er utnyttet.

Det er lett å identifisere teksten som en shellcode da jeg så noen kjente "op-codes", som xor (31) og cd 80 (unix' syscall: int 80).

31 db f7 e3 68 ff f4 f5 e2 68 fb f5
b0 f8 68 b0 fb fc ff 68 fc f5 e2 f5
68 f5 e2 b0 f6 68 e2 f5 fe f7 68 c6
f9 b0 e4 b9 90 90 90 90 31 0c 04 04
04 3c 1c 75 f7 89 e1 31 c0 b0 04 b2
1c cd 80 b0 01 cd 80

Derfor kan jeg anta at programmet er ment for ett UNIX operativ-system da vi ser nettopp den instruksjonen.

Man kan konvertere dette om til assembly manuelt ved hjelp av "X86 Opcode and instruction Reference" eller ved hjelp av ConvertShellcode.exe diskutert i denne blogg-posten av Lenny Zeltser. Når shellcoden er konvertert kan vi analysere koden ved hjelp av noe så enkelt som ett skrive-program eller om man vil gjøre dynamisk analyse av filen kan mann kompilere som vist i neste steg.

Slik ser maskinkoden ut når den er konvertert til assembly, og det er en liten programsnutt som dekrypterer en text-streng ved hjelp av en xor loop. På bildet ser vi hvilken "nøkkel" som verdiene skal utføres en logisk xor mot.. Jeg måtte ligge inn en "loop" funksjon i assembly koden, slik at JGE-instruksjonen hadde et sted å hoppe til om den IKKE er lik 28, så koden ser da slik ut:

På dette tidspunktet kan man gå mange forskjellige "veier" for å utføre videre analyse av filen. Man kan f.eks kompilere filen som vist på bildet under og kjøre den, enten direkte eller analysere den videre i en disassembler....

Men jeg følte for å koze meg litt til med utfordringen og lagde derfor ett Ruby-script som tar de krypterte dataene og xor'er de med nøkkelen som er 0x90909090. Når jeg først så disse verdiene i maskin-koden trodde jeg det var "nop"-instruksjoner men de viste seg å være verdien dataene skal utføre logisk xor med...

A computer infested with pest.

There is this PC i am working on at the moment which at first had ransom-ware installed, and got fixed by this guy who actually thought that to install a pirated version of Windows 7 was a good idea - and gave the owner back his machine.. A couple of days later the owner gave me a call (yeah after the malware was removed!!)  and had some issues with installing common programs so he could go back to everyday use of his PC.

After installing needed software and patching the OS, i found yet another malware-sample. This was clearly a old sample, the mothership had moved on as the sample could not contact certain domain. Without wanting to do so much more, i went on and installed software. 

So now, 30 days later i am sitting with a computer in my lap which are yelling for a Genuine Windows 7-version, and installed version is Win7PRO, key supports HomePremium OEM. To make things a bit more fun the recovery partition was also deleted during the "repair". Next time, please hit ALT+F10 during computer boot-process - at least try it - before deleting it! I am not sure how the process went when this computer got "fixed", and the latter example might not even work to remove some malware-samples.

It is not every day i create Windows USB-sticks so i decided to document this funny incident. 

Todo

1. Download proper windows version

Found this blog where i could download a windows 7 version: http://answers.microsoft.com/en-us/windows/forum/windows_7-windows_install/windows-7-home-prem-oa-acer-group/d6606520-8a45-4f47-a5ed-22978f4f602d

2. Create filesystem on stick

2.1. Create a ntfs partition with fdisk

start fdisk /dev/sdb, and i performed actions shown in picture below

2.2 Create NTFS filesystem.

Install ntfsprogs and ntfs-3g from repository

sudo mkfs.ntfs -f /dev/sdb1

3. Mount windows 7 iso and USB-stick.
Iso

sudo mount -o loop Downloads/X17-58996.iso /media/cdrom0/

Usb-Stick

sudo ntfs-3g /dev/sdb1 /media/usb0

4. Copy files from ISO to USB-stick

sudo cp -R /media/cdrom0/* /media/usb0/

4.5 Make a cup of tea and chill while it copy

5. Download ms-sys-*.tar.gz from http://ms-sys.sourceforge.net/#Download, unpack, compile and install.

In order for this to work i had to install gettext-package from repository and change dir to path of downloaded ms-sys*.tar.gz file before executing command below.

sudo apt-get install gettext; tar -xzvf ms-sys-*.tar.gz ; cd ms-sys*; make && sudo make install 

Next we can create a Windows 7 MBR on the device

sudo ms-sys -7 /dev/sdb

Unmount USB-stick, cross yer fingers while booting up the computer.

6. Create partitions 
Create backup partition (i decided to use 15GB)
Create OS partition

Make things easy

If one want to do this using GUI, i would believe that gparted package from debian-based linux distros could be used to format and create file-systems needed. To copy the files i would believe that unetbootin could be used as well. I have not tried this my self - so i can not confirm this work, but it does pretty much the same so why not.

Suspicious Domains

Malware are often trying to "phone home" to a command and control-server (C&C) or exploit a client by redirecting web-queries to a "malicious-landing-page", and often malware are relying on Domain Name System(DNS) to perform this action. By using DNS the botnet herder can move their mothership of listening C&C(s) quite easily, both by changing domain-names often to avoid detection of the IP-address of the mothership as well as well as move C&C to another node without upgrading all bots. Often these domains are created with fake credentials and used for a short period of time. Therefor one tell-tail to detect, not necessarily malicious network activity, but suspicious network activity, could be by obtaining whois information on the domain and especially "Created on" date. One could divide these results into a few "priority-levels" (1-3), where #1 is within a week, #2 is within a month and #3 within six months.

I made a tool to ease the analysis of a packet capture file (pcap) using Ruby and PacketFu-gem. The tool parses through network packets looking for DNS queries, if it receives such query it will do a simple Whois on the domain. If the domain is created within one of the three given levels shown above, it will be reported as a suspicious domain.

A simple test from a lab-environment is seen below, where we see two domains created this very week, that is a suspicious domain.

Of course, when we see the results it is quite easy to distinguish these two domain names from more common words, but the latter are often used as well - and this is at least one easy way to extract DNS queries and look for suspicious domain-names.... By using Snort a couple of days later on the same pcap-file i got confirmation on my findings, both alerted as "ET CNC Reported CnC Server IP".

I first saw this while doing analysis on one of the many different malware-samples shown in the "A botnets compromise" blog-series, where the different samples changed domain name frequently.