Security/Mentorships/MWoS/2014/Linux Audit heka plugin (Go)

Team

Introduction

We are a team of undergraduate students from Jaypee Institute of Information Technology who are passionate about Open Source and love developing something that can be useful to the community. We all are active member of Open Source Developers Club of our university. We got really excited when we first heard about MWOS, as it’s a great chance for us to work with such an amazing community. Having being contributed to open source organisation we know how it feels to develop software that will be used by the thousands of people and that is what motivates us towards MWOS.

Members

• Hardik Juneja
• Arun Sori
• Aalekh Nigam aalekhn
• Professor: Sanjay Goel
• Mozilla Advisor: Guillaume Destuynder

Project

Heka is a Mozilla project for logs routing, analysis, etc. (see http://hekad.readthedocs.org/en/latest/). Linux Audit logs are collecting various system calls and events in order to send them to a C user space program (auditd) over the netlink protocol. A Mozilla C plugin (https://github.com/gdestuynder/audisp-cef and https://github.com/gdestuynder/audisp-json) currently correlate, transforms, and send these events back to our logging architecture.

See also https://air.mozilla.org/mozilla-winter-of-security-audit-go-project/ for a complete presentation of the project.

Description

This project aims to deliver the same functionality as Linux Audit (auditd, audispd) + audisp-cef/json but in native Go as a plugin to Heka.

This means it will listen for events from the kernel via the Netlink protocol, parse the messages, convert them (to JSON using MozDef's native format), and pass them over to Heka.

Project Repository

Success Criteria

• Ability to process and forward audit events in pure Go from the kernel to Heka, as they would come out of audisp-json.

Roadmap

• Get more familiar with Go [August 18 2014]
• Work on communication using Netlink (receive audit messages from the kernel) [September 6 2014]
• Port the netlink code as Heka Plugin and start working with lua. [September 17 2014]
• Store messages in some structure/process them (lua) [September 24 2014]
• Correlate the kernel messages into a single mozdef-json message [September 31 2014]
• Send the message back to MozDef (can be done through Heka functions) over HTTPS [October 13 2014]

Updates

2014-08-11

Kick off meeting. (Etherpad Index) : https://etherpad.mozilla.org/YnR9hhqutn

Task for next 2 weeks :

• Get familiar with GO (first week)
• explore Netlink protocol, play with it and see how we can use it.

2014-08-25

(Etherpad Index)

Project Code

Task for this week:

• Write code that uses Netlink protocol.
• Receive Audit Messages from kernel.

2014-09-03

(Etherpad Index)

Project Code

Current Work:

• Wrote Netlink code for recieving audit messages (Code present in testing branch of the repository).
• Recieved invalid argument error while running the code.
• Problem probably due Header type argument.

Discussion Point:

• Cause of invalid Argument
• Debugging techniques in linux
• Problem in implementing strace

Upcoming Work:

• Take refererence from libaudit.h and insert some other proto decimal code
• Use strace tool to got through netlink execution
  • Use sudo -s and then strace -f go run <filename>
• Study strace and ltrace documentation

2014-09-08

(Etherpad Index)

Project Code

Current Work:

• Invalid Argument error removed :D, yay .
• Strace'd of auditd and netlink retrieved.
• Recieving nil as an output to audit message.

Discussion Point:

• Cause of nil error
• Way to parse audit message.
• Data type of audit message

Upcoming Work:

• Change the status code of proto, currently assigned 1000 status code is not correct.
• Output form of netlink message is structure.
• Use structure format same as audit_reply present in here libaudit.h
• Make a parse function like audit_get_reply as present in libaudit.h
• Study audit documentation and libaudit.h code

2014-09-15

Current Work:

• Added AuditRuleData Structure
• Added a AuditParse Function

Discussion Point:

• Cause of improper parsing.
• Follow the flow of auditd daemon from reading the code.

Upcoming Work:

• Structuring code to follow appropriate steps to enable Audit and receive response.
• Proper Parsing of a netlink response.

2014-09-22

• University Exams from 22 Sep to 26 Sep.
• Work will be resumed from 26 Sep.

2014-09-27

Current Work:

• Added AuditStatus Structure.
• Added a AuditSet Function.
• Appended Byte stream in netlinkMessage.

Discussion Point:

• Proper parsing of AuditStatus struct.
• Enabling Audit from our program.

Upcoming Work:

• Working AuditSet function
• Successfully Enabling Audit.
• Added AuditIsEnable Function.
• Moving current code to a go package.

2014-10-04

Current Work:

• AuditSet() Response successfully parsed into a AuditStatus struct.
• AuditIsEnabled Added.
• A netlinkAudit package.

Discussion Point:

• Adding Rules to audit.
• Receiving further responses from kernel.

Upcoming Work:

• Cleaning up old work; A simple driver program.
• Adding Rules to the Audit System.
• Adding Syscall information in a Rule struct.

2014-10-13

Current Work:

• AuditAddRuleData() Added.
• Adding rules to Audit kernel is successful.

Discussion Point:

• Remove hardcoded code for adding Rules.
• A JSON file like audit.rules.
• A way to map syscalls.

Upcoming Work:

• Moving Further ahead to a fully follow the auditd code flow.
• Adding AuditSetPid for registering our code with kernel.
• Adding Rules from a JSON File.
• Concurrent Way to run the code (Adding Channels).

2014-10-20

Current Work:

• X86 json added for mapping syscalls to a integers
• Changing design of the code.
• A new branch for work on the configuration part.

Discussion Point:

• Receiving response for the rules added
• Parsing the kernel response successfully.

Upcoming Work:

• A Getreply Function.
• Successful additon of rule from audit.rules.json file.
• Successful detection of the log message types and appropriate action.
• Diwali Holidays! More code on the way.

2014-10-28

Current Work:

• Successfully Parsing the responses received from kernel !!(Yay).
• Better Design; AutoGeneration of sequence numbers.
• More constants added.
• Mistakes in channel version. A simple version is added.
• Syscall mapping perfected.
• DeleteAllRules function added.
• Loading one or more rules from audit.rules.json.

Discussion Point:

• Addition of fields on the rules.
• Channel version of the getreply function.

Upcoming Work:

• A concurrent Getreply Function.
• A field mapping json file.
• More design changes.

2014-11-4

Current Work:

• A field map in JSON.
• Adding logs in a file.
• Channel Version added.
• Simplified Design.

Discussion Point:

• Working Field Addition and Error Handling.
• Porting code to work as Heka Plugin.
• Input type ( Http/Tcp/Udp ) for the Heka plugin.

Upcoming Work:

• A Final Field Version
• Prettify the output Messages.
• Moving on to the Heka Part.

2014-11-28

Current Work:

• A Final Field Version
• Prettify the output Messages.
• Moving on to the Heka Part.
• blocking points

Discussion Point:

• Creating Lpeg Grammer
• Audit-Heka Plugin discussion
• Use of LUa Decoder and Encoder

Upcoming Work:

• Lpeg Grammer
• Sample Decoder.

2014-12-03

Current Work:

• Lpeg Grammer
• Sample Decoder.

Discussion Point:

• Decoder Type
• Encoder Type

Upcoming Work:

• getPass for Golang
• CleanUp Golang Code
• Reconstruct Grammer

2014-12-15

Current Work:

• Lpeg Grammer
• Decoder Complete

Discussion Point:

• Decoder Errors

Upcoming Work:

• Cleanup Code
• Push to Mozilla Repository

2015-02-20

Presentation of the project available on AirMozilla https://air.mozilla.org/mozilla-winter-of-security-audit-go-project/

<date>

• current work
• blocking points
• discussion points
• upcoming work