How to Detect Memory Leaks Using Valgrind memcheck Tool?

       In order to handle memory related issues in C/C++ code, there is a most popular open source tool called as VALGRIND.
Along with Memcheck tool, there are several tools which are supplied to ease the debugging for the programmer.
Several Other Tools available with VALGRIND

• Memcheck is a memory error detector. It helps you make your programs, particularly those written in C and C++, more correct.
• Cachegrind is a cache and branch-prediction profiler. It helps you make your programs run faster.
• Callgrind is a call-graph generating cache profiler. It has some overlap with Cachegrind, but also gathers some information that Cachegrind does not.
• Helgrind is a thread error detector. It helps you make your multi-threaded programs more correct.
• DRD is also a thread error detector. It is similar to Helgrind but uses different analysis techniques and so may find different problems.
• Massif is a heap profiler. It helps you make your programs use less memory.
• DHAT is a different kind of heap profiler. It helps you understand issues of block lifetimes, block utilisation, and layout inefficiencies.
• SGcheck is an experimental tool that can detect overruns of stack and global arrays. Its functionality is complementary to that of Memcheck: SGcheck finds problems that Memcheck can’t, and vice versa..
• BBV is an experimental SimPoint basic block vector generator. It is useful to people doing computer architecture research and development.

Usage of Valgrind Memcheck Tool

The memcheck tool is used as follows :

valgrind --tool=memcheck ./a.out

As clear from the command above, the main binary is ‘Valgrind’ and the tool which we want to use is specified by the option ‘–tool’. The ‘a.out’ above signifies the executable over which we want to run memcheck.

To use this on our example program, test.c find below, try

gcc -o test -g test.c

This creates an executable named test. To check for memory leaks during the execution of test, try

valgrind --tool=memcheck --leak-check=yes --show-reachable=yes 

                                    --num-callers=20 --track-fds=yes ./test

Explanation about Command:

--tool = memcheck

--leak-check = Yes , It means find the memory leaks and display in the screen.

--num-callers =20, This will give stack trace, the functions which pushed in to stack

--track-fds = Yes, It will give info about socket Fds. It will give error  if sock fd is opened and same Fd is not closed

--show-reachable = Yes

 "Still reachable" means you haven't deallocated a block of memory before exiting, but had a pointer to it.

In a C++ program this means that some object could have not been deleted and therefore its destructor might not have been run and thus say some data might have not been saved onto disk for example and some other action might not have been taken and thus your program might produce unexpected behavior.

However there're no destructors in C programs, so your program just can't depend on that. Also deallocating memory takes some time, so by not freeing memory on exit you can save some time - your program will exit faster (this can be significant for programs with lots of data).

So IMO if your C program has "still reachable" blocks it's not a problem but this indicates that some code in the program doesn't free memory and so you can expect bugs when reusing that code.

How to save valgrind  output into separate file?

valgrind --log-file=<filename>

where <filename> is the file name for output. Later you can view this file with less or text editor.

 This tool can detect the following memory related problems :

• Use of uninitialized memory
• Reading/writing memory after it has been freed
• Reading/writing off the end of malloc’d blocks
• Memory leaks
• Mismatched use of malloc/new/new[] vs free/delete/delete[]
• Doubly freed memory

This outputs a report to the terminal like

==9704== Memcheck, a memory error detector for x86-linux.
==9704== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al.
==9704== Using valgrind-2.2.0, a program supervision framework for x86-linux.
==9704== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al.
==9704== For more details, rerun with: -v
==9704== 
==9704== 
==9704== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 11 from 1)
==9704== malloc/free: in use at exit: 35 bytes in 2 blocks.
==9704== malloc/free: 3 allocs, 1 frees, 47 bytes allocated.
==9704== For counts of detected errors, rerun with: -v
==9704== searching for pointers to 2 not-freed blocks.
==9704== checked 1420940 bytes.
==9704== 
==9704== 16 bytes in 1 blocks are definitely lost in loss record 1 of 2
==9704==    at 0x1B903D38: malloc (vg_replace_malloc.c:131)
==9704==    by 0x80483BF: main (test.c:15)
==9704== 
==9704== 
==9704== 19 bytes in 1 blocks are definitely lost in loss record 2 of 2
==9704==    at 0x1B903D38: malloc (vg_replace_malloc.c:131)
==9704==    by 0x8048391: main (test.c:8)
==9704== 
==9704== LEAK SUMMARY:
==9704==    definitely lost: 35 bytes in 2 blocks.
==9704==    possibly lost:   0 bytes in 0 blocks.
==9704==    still reachable: 0 bytes in 0 blocks.
==9704==         suppressed: 0 bytes in 0 blocks.

Let's look at the code to see what happened. Allocation #1 (19 byte leak) is lost because p is pointed elsewhere before the memory from Allocation #1 is free'd. To help us track it down, Valgrind gives us a stack trace showing where the bytes were allocated. In the 19 byte leak entry, the bytes were allocate in test.c, line 8. Allocation #2 (12 byte leak) doesn't show up in the list because it is free'd. Allocation #3 shows up in the list even though there is still a reference to it (p) at program termination. This is still a memory leak! Again, Valgrind tells us where to look for the allocation (test.c line 15).

test.c file

#include <stdio.h>

int main()
{
  char *p;

  // Allocation #1 of 19 bytes
  p = (char *) malloc(19);

  // Allocation #2 of 12 bytes
  p = (char *) malloc(12);
  free(p);

  // Allocation #3 of 16 bytes
  p = (char *) malloc(16);

  return 0;
}

More Info and examples can be found at:
http://www.thegeekstuff.com/2011/11/valgrind-memcheck/

What is NETFILTER in linux?

INTRODUCTION

Netfilter is a framework that provides hook handling within the Linux kernel for intercepting and manipulating network packets.

•Put more concretely, Netfilter is invoked, for example, by the packet reception and send routines from/to network interfaces.

•As the master Netfilter function is called with a packet, Netfilter runs through the list of registered hooks and calls the extensions in succession, which then handle packets as they desire.

 Background

•The actual Netfilter implementation is broken into two parts, the kernel portion known as Netfilter and the userland tool that interfaces with Netfilter and creates the rulesets, iptables. Both are required to implement your packet-filtering firewall.

•Netfilter includes support for IPv4 and IPv6. 

 Kernel Configuration

 In order to get started using Netfilter, you'll need to have your kernel compiled for Netfilter support.

 Most distributions include this support by default, so a quick test is in order. If you can insert the module ip_tables, then you won't need to worry about this section.

 As root, run the command

omodprobe ip_tables

 Then run

olsmod | grep ip_tables

  

If ip_tables show up, you're in good shape. Then rebuild a kernel. 

Functionality of NET FILTER:

 

 IP packet filter

 A packet filter is a piece of software which looks at the header of packets as they pass through, and decides the fate of the entire packet. It might decide to DROP the packet (i.e., discard the packet as if it had never received it), ACCEPT the packet (i.e., let the packet go through), or something more complicated.

 Why it is required:

oControl : when you are using a Linux box to connect your internal network to another network (say, the Internet) you have an opportunity to allow certain types of traffic, and disallow others.

oSecurity :Simply don't let anyone connect in, by having the packet filter reject incoming packets used to set up connections.

oWatchfulness.

 Packet Mangling:

 Packet mangling is the modification of packets at a packet-based network interface before and/or after routing.

 The process is sometimes used to prioritize network traffic by changing Type of Service (ToS) values in packet headers and to label a packet for a particular user space application.

 NAT

 In computer networking, network address translation (NAT) is the process of modifying IP address information in IP packet headers while in transit across a traffic routing device.

 Firewall

 A firewall is a device or set of devices designed to permit or deny network transmissions based upon a set of rules and is frequently used to protect networks from unauthorized access while permitting legitimate communications to pass

Netfilter has two groups of components, the kernel and user-mode pieces. The user-mode group consists of the iptables and related utilities, libraries, manual pages and scripts.

•

Netfilter—how it works

• Defines a set of hooks
• §Hooks are well defined point in the path of packets when these packets pass through network stack
• §The protocol code will jump into netfilter framework when it hits the hook point.
•  Registers Kernel functions to these hooks:
• §Called when a packet reaches at hook point
• §Can decide the fate of the packets
• §After the functions, the packet could continue its journey
• §Five hooks defined in IPv4: PRE_ROUTING, LOCAL_IN, FORWARD, LOCAL_OUT, POST_ROUTING.
• §Each hook can alter packets, return NF_DROP, NF_ACCEPT, NF_QUEUE, NF_REPEAT or NF_STOLEN.

•

 

  

•

1. Whenever a packet receives to kernel, the first hook which will be called is "PRE_ROUTING" Hook.

  2.  After that there is a router which will decide the fate of the packet, it means based on the destination IP address of the packet, it will decides whether the packet belongs to local(it will be given to Application space, receive these packets through UDP sockets in appl) or Non Local packets(Through Post Hook it will be send out).

  
3. To send the Non Local packets out via post Hook, the Linux machine should become router. It means the Forward hook should be enabled(ipv4.ip_forward =1)

       vim /proc/sys/net/ipv4/ip_forward

   change this ip_forward =0 to 1

   echo 1 > /proc/sys/net/ipv4/ip_forward

  

  4.  The Local generated packet from the application space, will be traversed via LOCAL_OUT and POST_HOOK.

e.g: If you ping to some machine, then the packet will be locally generated and go via LOCAL_OUT and POST_HOOK.

   5.  If the packet is destined to same Machine(Application Space of same Machine),  Then Packet will be traversed via PRE_HOOK and LOCAL_IN. 

Here no need to register a LOCAL_IN hook, in PRE_HOOK just do NF_ACCEPT , then all packets can be received in APPLICATION SPACE using UDP sockets.

Some Interesting Figures:

 

 

 

•

 

PRE_ROUTING  :  §Incoming packets pass this hook in ip_rcv() before routing

LOCAL_IN : § All incoming packets addressed to the local host pass this hook in ip_local_deliver()

FORWARD : § All incoming packets not addressed to the local host pass this hook in ip_forward()

LOCAL_OUT: §All outgoing packets created by this local computer pass this hook in ip_build_and_send_pkt()

POST_ROUTING: All outgoing packets (forwarded or locally created) will pass this hook in ip_finish_output()

•

•

 The netfilter function has five possible return values:  § 

NF_ACCEPT : continue callback chain

§NF_DROP     : drop the packet and stop the chain

§NF_STOLEN : stop the chain

§NF_QUEUE  : send the packet to userspace

§NF_REPEAT : call the hook again 

 This program will send packet to loop back interface, In kernel it will go to
APPLICATION CODE -> LOCAL_OUT -> POST_HOOK -> PRE_HOOK ->LOCAL_IN -> APPLICATION CODE

User SPACE Code -> compile  < c++ filename.cpp -lpthread -o test.exe>

#include <stdio.h>
#include <iostream.h>
#include <arpa/inet.h>
#include <string.h>

#define BUFFSIZE 5096

int sendlen, receivelen;
int i,count, sentCnt=0;
unsigned char buffer[BUFFSIZE];
/* client Socket */
struct sockaddr_in receivesocket;
/* Server Socket */
struct sockaddr_in sendsocket;

/* Sock FD */
int sockFd;

/* Number of Times you need */
unsigned int ch;
unsigned int noOfTimes;

/* Send UDP Data */
int sendUDPData();

/* Receive Call Back Function */
void *recvNetfilterData(void *);


int main(int argc, char *argv[]) {
int ret = 0;

   /* Create the UDP socket */
   if ((sockFd = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP)) < 0) {
           perror("socket");
           return -1;
   }

   /* my address or Parameters ( These are required for Binding the Port and
        IP Address )
     Bind to my own port and Address */
   memset(&receivesocket, 0, sizeof(receivesocket));
   receivesocket.sin_family = AF_INET;
   receivesocket.sin_addr.s_addr = htonl(INADDR_ANY);
   receivesocket.sin_port = htons(2905);

   receivelen = sizeof(receivesocket);

   /* Bind the my Socket */
   if (bind(sockFd, (struct sockaddr *) &receivesocket, receivelen) < 0) {
           perror("bind");
           return -1;
   }

   /* Server address or Parameters , Sending System Address and Port Num */
   memset(&sendsocket, 0, sizeof(sendsocket));
   sendsocket.sin_family = AF_INET;

   /* give Proper IP address */
   sendsocket.sin_addr.s_addr = inet_addr("127.0.0.1");
   sendsocket.sin_port = htons(2905);

   /* Create Seperate Thread */
   /* Start the Receiving Thread */
   pthread_t threadId;
   if(pthread_create(&threadId, NULL, recvNetfilterData, NULL))
   {
       cout<<"Error in creating receiver thread";
       return 0;
   }


   /* Do Loop -> Send UDP Data */
   do
    {
       cout<<endl;
       cout<<" Enter your choice:\t"<<endl;
       cout<<" 1. Send UDP Data" <<endl;
       cout<<" 2. exit" <<endl;
       cin>>ch;
       cout<<endl;

       switch(ch)
       {

           case 1:
                   cout<<"Enter the Length of the Payload "<<endl;
                   cin>>sendlen;
                   cout<<"Enter How many times you want to send data "<<endl;
                   cin>>noOfTimes;
                   /* Send UDP Data */
                   sendUDPData();
                   break;

           default:
                  cout<<"Invalid Choice\n";
                  break;
       }
 }while(ch!=2);
return 0;
}
/*
 *  sendUDPData
 */
int sendUDPData()
{
        int count=0;
        memset(buffer, 31, sendlen);

        for(count=0; count< noOfTimes;  count++)
        {
           /* Send the UDP Data */
           if (sendto(sockFd, buffer, sendlen, 0,
     (struct sockaddr*)&sendsocket, sizeof(sendsocket)) != sendlen)                   

           {
                perror("sendto");
                return -1;
           }
           else
           {
                sentCnt++;
           }
         }
    return 0;
}
/*
 * Dump Data
 */
void dumpData(unsigned char *data,  unsigned int len)
{
    unsigned int uIndx;

     if(data)
    {
        for(uIndx=0; uIndx<len; ++uIndx)
        {
           if(uIndx%32 == 0)
           {
              printf("\n%4d:", uIndx);
           }
           if(uIndx%4 == 0)
           {
                 printf(" ");
           }
           printf("%02x", data[uIndx]);
        }
    }
    printf(" Length of Bytes: %d\n", len);
    printf("\n");
}

/*
 * recvNetfilterData()
 */
void *recvNetfilterData(void *args)
{
    unsigned char buf[5096];
    int receivedLen = 0;

    while(true)
    {
        memset(buf, 0, BUFFSIZE);
        /* Recieve the Data from Other system */
        if ((receivedLen = recvfrom(sockFd, buf, BUFFSIZE, 0, NULL, NULL)) < 0)
        {
  perror("recvfrom");
                return 0;
        }
        else if(receivedLen == 0)
        {
             cout<< " The Return Value is 0";
        }
        else
        {
              /* Print The data */
             cout<< " Recvd Byte length" << receivedLen <<endl;
             dumpData(buf, receivedLen);
        }
    }
}

Kernel SPACE Code -> Filename : muniTestNetfilter.c

Makefile

obj-m += netfil.o
netfil-objs :=  muniTestNetfilter.o
all:
        make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
        make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/moduleparam.h>
#include <linux/version.h>
#include <linux/netlink.h>
#include<linux/udp.h>

/* Optional Headers  not Required, If you don't want Remove*/
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/socket.h>
#include <linux/icmp.h>

/* These are Required for NETFILTER functionality */
#include <net/checksum.h>
#include <linux/netfilter.h>
#include <linux/netfilter_ipv4.h>
#include <linux/interrupt.h>


#define DRIVER_AUTHOR_NAME  "MUNI"
#define DRIVER_DESCRIPTION  "NETFILTER"

#define DEST_PORT_NUM      2905

/* NETFILTER*/
static struct nf_hook_ops test_hookPreRouting;
static struct nf_hook_ops test_HookPostRouting;
/*
 * preRoutingHookEntryFunc -> Pre Routing Function
 */
static unsigned int preRoutingHookEntryFunc(

                       unsigned int unHookNum,
                       struct sk_buff **skb,
                       const struct net_device *inDevice,
                       const struct net_device *outDevice,
                       int (*okfn)(struct sk_buff *))
{
    /* Copy the Socket Buffer */
    struct sk_buff *sockBuff = *skb;

    /* Intialise  the UDP Header  */
    struct udphdr *uh = NULL;

    /* If the IP hdr is Zero or SockBuff is Zero Just Accept */
    if(NULL == sockBuff || NULL == sockBuff->nh.iph)
    return NF_ACCEPT;

    /* Point to the UDP Header */
    uh = (struct udphdr*)(sockBuff->data + (sockBuff->nh.iph->ihl * 4));

    /* Filter only UDP Packet */
    if((sockBuff->nh.iph->protocol == IPPROTO_UDP) &&
                       (ntohs(uh->dest) == DEST_PORT_NUM))
    {
        /* Retrieve Udp Header */
        printk(KERN_INFO " Recived Packet [ PRE HOOK ] with  IPDest:%x:PortNum:%d"
                "IPSrc%x:PortNum:%d\n",
               ntohl(sockBuff->nh.iph->daddr),ntohs(uh->dest),
               ntohl(sockBuff->nh.iph->saddr),ntohs(uh->source));
    }
   /*  All  Packets Just NF_ACCEPT
    *  If it is LOCAL packets then it will go to LOCAL_IN 

    *  to Application else for NON LOCAL Packets it will go 

    *  POST HOOK and Out
    */
    return NF_ACCEPT;
}
/*
 * postRoutingHookEntryFunc -> Post Routing Function
 *
 * POST HOOK will be called for NON LOCAL packets from 

 *  PRE_HOOK- POST HOOK via FORWARD
 * else LOCAL_OUT - POST HOOK for Local Generated Packets from the system(PING)
 *
 */
static unsigned int postRoutingHookEntryFunc(
        unsigned int unHookNum,
        struct sk_buff **skb,
        const struct net_device *inDevice,
        const struct net_device *outDevice,
        int (*okfn)(struct sk_buff *))
{
   /* Copy the Socket Buffer */
    struct sk_buff *sockBuff = *skb;

    /* Intialise  the UDP Header  */
    struct udphdr *uh = NULL;

    /* If the IP hdr is Zero or SockBuff is Zero Just Accept */
    if(NULL == sockBuff || NULL == sockBuff->nh.iph)
    return NF_ACCEPT;

    /* Point to the UDP Header */
    uh = (struct udphdr*)(sockBuff->data + (sockBuff->nh.iph->ihl * 4));

    /* Filter only UDP Packet */
   if((sockBuff->nh.iph->protocol == IPPROTO_UDP) &&
                     (ntohs(uh->dest) == DEST_PORT_NUM))
    {
        /* Retrieve Udp Header */
        printk(KERN_INFO " Recived Packet In [ POST HOOK ]  IPDest:%x:PortNum:%d"
                "IPSrc%x:PortNum:%d\n",
               ntohl(sockBuff->nh.iph->daddr),ntohs(uh->dest),
               ntohl(sockBuff->nh.iph->saddr),ntohs(uh->source));
    }

    /*  All  Packets Just do NF_ACCEPT
     *  then packets  will go Out of the Driver
    */
    return NF_ACCEPT;
}
/*
 * Module Init Function
 */
static int __init muni_test_netfilter_init(void)
{
    int unReturn = 0;

    printk(KERN_INFO "Loading MUNI NETFILTER Module\n");

    /* Intialisation of Pre Hook Parameters */

    /* Which Look You need to Call */
    test_hookPreRouting.hooknum  = NF_IP_PRE_ROUTING;
    /* Type of Family */
    test_hookPreRouting.pf       = PF_INET;
    /* Call back function for Prehook */
    test_hookPreRouting.hook     = preRoutingHookEntryFunc;
    /* Priority means, Which module receives the packet first
    * Here Priority = FIRST, means this module receives the packet first.
     */
    test_hookPreRouting.priority = NF_IP_PRI_FIRST;

    /* Register the Pre Routing Hook in Kernel */
    if((unReturn = nf_register_hook(&test_hookPreRouting)) < 0)
    {
        printk(KERN_INFO "Registration of Pre Routing"
                                " Hook is failed and ERROR=%d\n", unReturn);

        /* Un Register the Pre Routing Hook */
        nf_unregister_hook(&test_hookPreRouting);
        return unReturn;
    }

    /* Intialisation of Post Hook Parameters */
    test_HookPostRouting.hooknum = NF_IP_POST_ROUTING;
    test_HookPostRouting.pf      = PF_INET;
    /* Call back function for Post Hook  */
    test_HookPostRouting.hook    = postRoutingHookEntryFunc;

    /* Priority means, Which module receives the packet first
     * Here Priority = FIRST, means this module recives the packet first.
     */
    test_HookPostRouting.priority= NF_IP_PRI_FIRST;

    /* Register the Post Routing Hook in Kernel */
    if((unReturn = nf_register_hook(&test_HookPostRouting)) < 0)
    {
        printk(KERN_INFO "Registration of Post Routing"
                                "Hook is Failed and ERROR=%d\n", unReturn);
        /* Un Register the Post Routing Hook */
        nf_unregister_hook(&test_HookPostRouting);
        return unReturn;
    }

    return 0;
}

/*
 * Module Exit Function
 */
static void __exit muni_test_netfilter_exit(void)
{
    printk(KERN_INFO "UnLoading MUNI NETFILTER Module\n");

    /* Unregister the Pre Routing Hook */
    nf_unregister_hook(&test_hookPreRouting);

    /* Unregister the Post Routing Hook */
    nf_unregister_hook(&test_HookPostRouting);

}
/* Init Module */
module_init(muni_test_netfilter_init);
/* Exit Module */
module_exit(muni_test_netfilter_exit);

/* Some MODULE PARAMETERS */
MODULE_LICENSE("COPY");
MODULE_AUTHOR(DRIVER_AUTHOR_NAME);
MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
MODULE_VERSION("2.6.19.184");

Note:

• The Same user Space Code can be modified to send as Non Local Packet. This can happen only  by changing the IP address of the server socket and also this code should run in different machine.
• No need to explicitly declare the LOCAL_IN and LOCAL_OUT hooks, kernel will take care.

•

•