Python - Basics- Learn the Python with Examples

In this post, you can find the basics of python.

Run "python" from any Linux terminal (or shell) and practice all examples mentioned below.

-------------
Operations:
--------------

+, -, %, *

2**3 = 2 power 3 = 8

--------
Varibles
--------

Variables is used to store the values.

x=6
y=8

print (x+y)


How to read from keyboard
-------------------------

x=input("enter the Number here ')

print x


Modules and functions
----------------------

x=abs(16)

y=pow(2)

How import to module
=================

import math

The respective module can be accessed by using <modulename>.function

math.floor(18.7) = 18

-------
Strings
========

strings can be stored as " " or ' '

But if there is a string like 'he's muni' -> Error

to avoid these kind of issues then use "he's muni"

Escape Character (\)
-------------------
'he\'s muni' -> it is going to escape next character.

another example
'bucky said, "hey now " to me' -> Solution is: 'bucky said, \"hey now\" to me'

Joining Strings or Concatenation of strings (+)
============================================

a= "muni"
b= "sekhar"

print a + b => munisekhar

'+' sign is used to concatenate the string or Join the strings.


How to concatenate the string and integer
-----------------------------------------

num = 18
print "hello world " + num  -> Will lead in to error.

Since the integer can be concatenate with the string, so we need to convert
the integer in to string.

1 way : str(num)  -> convert num to string.

print "hello world" + str(num)

2nd way: `num`

print "hello world" + `num`

3rd way: repr(num)

print "hello world" + repr(num)


Difference between raw_input vs input
-------------------------------------

raw_input-> Generally used for string. It takes input and converts in to string

input -> Generally used for math(numbers), it can't be used for strings

ex:

bucky = input("Enter the number")
print bucky

>> After execute: Enter the number
>>> muni

--------> lead to error.

Correct way of doing is:
-----------------------
bucky = raw_input("Enter the number")
print bucky

Enter the number
muni

output : muni

=====================
Sequences and Lists
=====================


The most basic datastructure in python is sequence - Each element in a sequence is
assigned with number - It's position or index.
index starts from zero..

The list is a most versatile datatype available in Python which can be written as a list of
comma-separated values (items) between square brackets.
Important thing about a list is that items in a list need not be of the same type.

lists:
-----

list1 = ["muni", "sekhar", 12, 34]

print list1[0]

output: muni

print list[3]

output: 34

How to print the end of the list: It always start from -1.
---------------------------------
print list[-1]  

output: 34

strings indexing
-----------------

str = "muni"

print str[0]   -> m
      str[1]   -> u


==========
slicing
=========

slicing can be done using ':' (colon) operator

example = [0,1,2,3 4,5,6,7,8, 9]

print example[0:4]

output: 0,1,2,3

It explains as include start of the element and print before the end (Here 4 is not included).

example = [0,1,2,3 4,5,6,7,8, 9]

print example[1:9]

[1,2,3,4,5,6,7,8]

How to include the last element
-------------------------------
print example[1:10]

or

print example[1:]

[1,2,3,4,5,6,7,8,9]


print example[:9] = [0,1,2,3,4,5,6,7,8,9]


How to print the element by adding 2
-------------------------------------

print example[1:9:2]  -> it increment every digit by 2.

[1,3,5,7]


print example[-5:9:2]  -> it increment every digit by 2.

[5,7]

Last digit is -1 i.e. 9

print example[-1:-4:-2]  -> it increment every digit by 2.

[9,7]

Double colon Operator
----------------------

example[::-2]
[9, 7, 5, 3, 1]

example[::2]
[0, 2, 4, 6, 8]

===================
Editing Sequencing
===================

The list can be concatenated using '+' operator

list1 = [1,2,3,4]
list2 = [5,6,7,8]

list 3 = list1 + list2

print list3

output : [1,2,3,4,5,6,7,8]

Mutiplication
--------------
num= 20

print num*2

Output -> 40

Double the list (repeat the number)
------------------------------------

num = [20]
print num*2

output -> [20,20]


list3*2 = [[1,2,3,4,5,6,7,8, 1,2,3,4,5,6,7,8]

===================
Using "in" Keyword
===================
"in" keyword is used to check whether the value or character present in the
list or string.

example 1:
---------

name = "muni"

if('r' in name) :
print "true"
else :
print "false"

->
Output : false

name = "muni"
if('m' in name):
  print "true"
else:
  print "false"

->
Output : true

example 2:

list = [1,2,3,4,5,6,7,8,9]
if(10 in list):
  print "true"
else:
  print "false"

->
Output : false

====================================
Some of the Useful List Functions
====================================

list1 = [1,2,3,4,5,6,7,8]

length function
----------------

print len(list1)

->
Output : 8

Get the Maxium value of the list
--------------------------------
print max(list1)

->
Output : 8

Get the Minimum value of the list
--------------------------------
print min(list1)

->
Output : 1

Change the List sequence number to different one
-------------------------------------------------

list1[3] = 10

print list1

->
Output : [1,2,3,10,5,6,7,8]

How to del particular number in a list(Use del keyword)
-------------------------------------------------------

del list1[3]

print list1

->
Output : [1,2,3,5,6,7,8]

-----------------------------
Convert the string in to list
-----------------------------

str = "bucky"

print list(str)

output:

['b','u', 'c','k','y']

===============
Slicing Lists
===============

example = list("bucket")

print example

Output->
['b', 'u', 'c', 'k', 'e', 't']

Append or replace the sequence of strings
==========================================
The below will replace letters from index=3 onwards

example[3:] = list("munisekhar")

print example

['b', 'u', 'c', 'm', 'u', 'n', 'i', 's', 'e', 'k', 'h', 'a', 'r']


Appending the list
===================

Below example append the list from position 1 to after 1

example = [0,1,2,3,4,5,6,7]

example[1:1] =[3,3,3]

print example

->
Output = [0,3,3,3,1,2,3,4,5,6,7]


example = [0,1,2,3,4,5,6,7]

example[2:4] = [10,11,12]

print example

->
Output = [0,1,10,11,12,4,5,6,7]



How to delete the list elements using Slicing operator
======================================================

example = [0,1,2,3,4,5,6,7]

example[1:5]   means starts from 1 to 4 (before 5)

example [1:5] = []

print example

->
Output = [0,5,6,7]

=========================
Introduction to methods
==========================

object.actionfunction(arguments)

for example:

myface.punch(hand) -> myface is the object
                     punch is the action
                     hand is the argument.


fruitslist = ['apples', 'jackfruit', 'banana', 'guava']

append
-------
is used to add the 'orange' string at the end of the list.
------
fruitslist.append('orange')

->
output:
>>> fruitslist = ['apples', 'jackfruit', 'banana', 'guava']
>>> fruitslist.append('orange')
>>> print fruitslist
['apples', 'jackfruit', 'banana', 'guava', 'orange']

-----
count  : How many times the element (string) present in the list.
-----
fruitslist.count('apples')

-------
Output
->
1

--------
extend
--------
one = [1,2,3]
two = [4,5,6]

one.extend(two)

print one

output->
The extend will take two list as a paramter and added to the one list.
[1,2,3,4,5,6]


Some more functions
-------------------

say =['muni', 'sekhar', 'reddy', 'a']

index  : to get the exact index number
======
say.index('reddy')

print say

output
->
-------
2

------
insert  : you can insert the data at particular index of list
------
say.insert(2, 'rishi')

print say

->
output:
['muni', 'sekhar', 'rishi', 'reddy', 'a']

----
pop  : remove the data using index
----

say.pop(2)

print say

->
output:
['muni', 'sekhar', 'reddy', 'a']

-------
remove  : pass the exact data to remove from the list
-------

say.remove('a')

print say

->
output:
['muni', 'sekhar', 'reddy']

--------
reverse  : reverser the list
--------
say.reverse()

->
output:
['reddy', 'sekhar', 'muni']

==================
Sort and Tuples
==================

Sort -> Using sort function, the list can be sorted.

new = [33, 5, 4,66,9,10]

new.sort()

print new

->
output:
[4,5,9,10,33,66]


String can be sorted using "sorted" keyword
============================================
name="muni"
sorted(name)

->
output:
['i','n','m','u']

-------

Tuples
======

For tuples , we can add or insert or modify the values in the tuples.

But whereas in list, the values can be modified.

buckytuple = (22,33,44,56)

bucky[2] = 44

tuples can't be modified.


================
String n Stuff
================

There are many useful string techniques, some of them are:

How to add your add content in the strings  (Using % operator)
------------------------------------------

bucky = " hey there %s, How are %s"

declare a tuple
----
var = ('ramu', 'sita')

The own content can be added using "%" operator.

print bucky % var

->
output:
hey there ramu, How are sita
-----------------------------------------
The same can be changed as:

varc = ('ra', 'si')

print bucky % varc

->
output:
hey there ra, How are si


Find Method for strings
========================

bucky = "are you there muni" , i am in cgiri"

bucky.find('muni')
->
output  : 14

'm' is at position of 14.


Some more String methods:
=========================

join the strings -> Using join keyword
----------------
sequence = ['hey' , 'there', 'boss']
glue = 'hoss'

glue.join(sequence)

output->
heyhosstherehossbosshoss

How to lower case the string (Using lower keyword)
----------------------------

randstr = "I am WORK women"
randstr.lower()

output->
i am work women

How to replace a string ( Using replace keyword)
------------------------
samplestring = " I am munisekhar Reddy"

samplestring.replace('Reddy', 'naidu')

Output-> I am munisekhar naidu



DICTIONARY
===========

dictionary is collection of keys and values.

" The values can be stored against the words"

syntax:
sampledictinary = {key:values}

sampledictionary = {'dad':'munireddy', 'mother':'aruna', 'grandmother':'gangamma'}

sampledictionary['dad']

output-> 'munireddy'

Some of the useful methods of dictonaries
------------------------------------------

clear method
-----

sampledictionary.clear()

clear the dictionary content and make the dictionary as empty.

output->
{}

Copy the dictionary
--------------------
anotherdict=sampledictionary.copy()

output->
anotherdict = {dad':'munireddy', 'mother':'aruna', 'grandmother':'gangamma'}

Check the dictonary has key or not
----------------------------------

sampledictionary.has_key('sister')

output->
false

sampledictionary.has_key('dad')

output->
true

=============
IF statement
=============

syntax:

if() :  (After end of if there is colon operator needs to give here)
  <indent> print sample
else :
  <indent> print else.



Sample example->

tuner = "blackcolour"
if(tuner == "blackcolor"):
   print " I am in IF loop"
else :
   print " I am in else loop"

=======================
IF/ELIF/ELSE statement
=======================
tuner = "blackcolour"

if(tuner == "blackcolour"):
  print " I am in IF loop"
elif (tuner == "redcolour"):
  print " I am in  red loop"
elif (tuner == "yellow"):
  print " i am in Yellow"
else :
  print " I am in else loop

=====================
NESTED IF statements
=====================
tuner = "redcolour"
animal = "pig"

if(tuner == "blackcolour"):
  print " I am in IF loop"

  /* Nested IF statements, IF statements within IF */
  if animal == pig:
     print " i am in pig"
  else :
     print " i am not pig"

elif (tuner == "redcolour"):
  print " I am in  red loop"

elif (tuner == "yellow"):
  print " i am in Yellow"

else :
  print " I am in else loop


=======================
Comparision Statements
======================

is and in statements in IF statements
-------------------------------------

in operator   ( It checks whether that element is present in the string)
------------

example = "pizza"

if 'p' in example:
  print true
else :
  print false

Output-> true.


is operator
------------

Whether it is same object or not.

list1 = [120,20]
list2=[120,20]

if (list1 is list2):
  print true
else :
  print false

Output-> false

since it is not same object.

example 2
----
list1 = list2= [120,20]
if (list1 is list2):
  print true
else :
  print false

Output-> true

=====================
And and Or operators
=====================
and -> Used for Range operation checks
or  -> either one is true


e.g:

example =3

if example<4 and example<6 :
  print "The AND condition is true"
elif example>19 or example<23:
  print "The OR condition is true"
else:
  print "nothing is matching"

====================
FOR and WHILE Loops
====================

While Loop example
------------------

b=1
while b<10:
  print "count:" + b
  b= b+1

Output-> count: 1,2 3,4 5,6,7,8,9


For Loop example
-----------------

fruitlist= ["milk", "juice ","butter"]
for food in fruitlist:
  print "I want " + food

Note: food will be pointing to the staring element of the list.
      There is no need to assigment of this variable also no need of increment.

 

Output->

I want milk
I want juice
I want butter

==================================
Infinite Loops and break statement
===================================

Let's have a dictionary which stores the values against the keys.

ages = {'dad':24, 'mom':45, 'sister':18}

for item in ages:
 print item

Output->
  item will be the keys of the dictionary.

dad
mom
sister


Then How to get the values of the dictionary
--------------------------------------------

ages = {'dad':24, 'mom':45, 'sister':18}

for item in ages
 print item, ages[item]

Output:

  dad    24
  mom    45
  sister 18
---------------
Infinite Loops
--------------

while 1:
  num=input("Enter the number")
  if num == 5:
    print "exit"
    break


example 2:
----------

while 1:
  name= raw_input("Enter the name")
  if(name =='quit'):
    break
 
Output:
Enter the name muni
Enter the name sekhar
Enter the name reddy
Enter the name quit


======================
Building functions
======================

How to write your own functions:

Use the keyword as "def" before the function name (def is called as definition).

function syntax:

def funcname(<parameters>):
  do some logic
  return parmeter

---------
example 1:
----------
def firstfunction(x):
 return "I want " +x

print firstfunction('cofee')
print firstfunction('Tea')
print firstfunction('milk')

Output->
I want cofee
I want Tea
I want milk


---------
example 2:
----------

def firstfunction(x):
 return  x+10

print firstfunction(10)
print firstfunction(20)
print firstfunction(30)

Output->
I want 10
I want 20
I want 30


==============================
Default parameters in function
==============================

def samplefunc(first= 'muni', last='sekhar'):
  print '%s %s' % (first, last)

firstfunction()

Output-> muni sekhar

In the above example, there are default values are assigned to the parameters.

function is called with out arguments. It prints the default values.

ex 2:
----
one can also override the default parameters as:

def samplefunc(first= 'muni', last='sekhar'):
  print '%s %s' % (first, last)

firstfunction('raj', 'reddy')

Output-> raj reddy


ex 3:
----
You can override the one parameter as:

def samplefunc(first= 'muni', last='sekhar'):
  print '%s %s' % (first, last)

firstfunction(last='reddy')

Output-> muni reddy

=====================
Multiple arguments
=====================

How to accept multiple arguments using single variable in a function

Use asterik (*)

ex 1:
-----

def samplefunc(*food):
  print food

samplefunc('apples', 'banana', 'cherrys')

Output->
(apples', 'banana', 'cherrys',)

Here the '*' infront of variable accept the multiple parameters (as tuple).


ex 2:
-----

def samplefunc(name, *ages):
  print name
  print ages

samplefunc('muni', (22,33,44,55))

Output->
muni, ((22, 33,44,55),)


=============================================
How to convert the parameters as dictionaries
==============================================

Double asterik (**) will do the magic.

example 1:
---------

def cart(**items):
  print items


cart(apple=40, grapes=65, strawberryies=89)

output-> Convert in to dictionaries
   {'apple':40, 'grapes':65, 'strawberryies':89

Note :apple=40 : equal sign makes key = value by compiler.

example 2:
---------
def cart(first, last, *ages, **items):
  print first
  print last
  print ages
  print items


cart('muni', 'sekhar', 22,33,44,55, apple=40, grapes=65, strawberryies=89)

equal sign  makes the compiler to think as dictionary.

output->  
muni
sekhar
(22,33,44,55)  (tuple)
{'apple':40, 'grapes':65, 'strawberryies':89} (dictionary)

==================================
Tuple and Dictionary as parameters
==================================

* (single asterik) is used for tuple
** (double asterik) is used for dictionary.


tuple as paramter:

--------------------------------------
def tupleparameter(a, b,c):
 return a+b*c

sampletuple = (1,2,3)

print tupleparameter(*sampletuple)

Output -> 7

Here a takes value a=1, b=2, c=3
-----------------------------------

dictionary as paramter:
----------------------

def dict(**example):
 print example

sampledict = {'dad':45, 'mom':55, 'sekhar':60}

dict(**sampledict)

===================
CLASSES and SELF
===================
 class contains data members and method.

 In methods the first parameter is "self" followed by parameters,

self
----
 self is the temporary holder for object name. The respective object is stored in self.

class syntax:
-------------
class <classname>:
 

  def method1 (self, x, y):
    statements

  def method2 (self, x, y):
    statements
 

#how to create object.
firstobj = classname()

How to access the method using the object?
-----------------------------------------
firstobj.method1()
firstobj.method2()


--------
example 1:
---------

class firstclass:
 def create(self, name, salary):
    self.name = name
    self.salary=salary

 def display(self):
    print "name : %s and salary: %d" % (self.name, self.salary)

# create object
firstobj= firstclass()
secondobj = firstclass()
firstobj.create('sekhar', 20)
secondobj.create('muni', 10)

firstobj.display()
firstobj.display()

============================
Super Class and Sub Classes
============================

class parentclass:
  var1="method"
  var2 ="methos2"
----------------------------------------
syntax: class derivedclass(parentclass):
----------------------------------------
#derived class
class derivedclass(parentclass):
  pass  (no definition or NULL statement, definition can be written later).


pass keyword:
------------
In Python programming, pass is a null statement. The difference between a comment and pass statement in Python is that, while the interpreter ignores a comment entirely,
pass is not ignored. But nothing happens when it is executed. It results into no operation (NOP).

---------------------
parentobj= parentclass()
print parentobj.var1

derivedobj= derivedclass()
print derivedobj.var1

---------
example :
---------
class parentclass:
  var1="first variable"
  var2="second variable"

class derivedclass(parentclass):
  pass #no definition

parentobj= parentclass()
print parentobj.var1
print parentobj.var2

derivedobj=derivedclass()
print derivedobj.var1
print derivedobj.var2
`

output->
first variable
second variable

first variable
second variable

=====================================
Overwrite Variables in derived class
=====================================
Python provides a flexibity to overwrite variables in the derived class.

class parentclass:
 var1="muni"
 var2="sekhar"

class derived(parentclass):
  var2="reddy"

pobj = parentclass()
print pobj.var1
print pobj.var2

print "derived class"
cobj = derived()
print cobj.var1
print cobj.var2

Output->
muni
sekhar

derived class
muni
reddy


=========================
Multiple Parent Classes
=========================

In python the derived class can inherit the properties from multiple classes.

Suppose if you have two classes  Mom and dad, the derived class (child) can inherit from both classes.


class Mom:
   var1 = "hey i am mom"

class Dad:
   var2 = "hey i am dad"

class child (Mom, Dad):
   var3 = "hey i am child"

momobj = Mom()
dadobj = Dad()
childobj  = child()

print childobj.var1
print childobj.var2
print childobj.var3


Output->
hey i am mom
hey i am dad
hey i am child


===============
Constructors
===============
Constructor is used to intialise the object data members of a class.

Construtor is called when object is created for class.
--------------------------------------------------------
syntax for the constructor
==========================
def __init__(self):

--------------------------------------------------------

class parentclass:

  def __init__(self):
    print "constructor is called"
    print "constructor is called when object is created"

pobj= parentclass()

Output->
constructor is called
constructor is called when object is created

===============
Import Modules
===============

python import the modules using the keyword called "import"

syntax:
-------

import <filename>

-> How to Access:

filename.<function>()


example :
---------

Just a sample file "testmodule.py"

testmodule.py
--------------
def testmod():
  print " I am in testmodule"

sample.py
----------
import testmodule   >>> Import the testmodule

testmodule.testmod()


Output->
I am in testmodule



===============
reload Modules
===============

"reload" is the keyword which can be reload the module.
Used if there are any modification done on the module.

example:
--------

Just a sample file "testmodule.py"

testmodule.py
--------------
def testmod():
  print " I am in testmodule"

sample.py
----------
import testmodule   >>> Import the testmodule

testmodule.testmod()

samplevariable= testmodule.testmod

samplevariable()

Note: The testmodule.testmod can be stored in the normal variable and can be called.


In case if we modify ->

testmodule.py
--------------
def testmod():
  print " I am in sample module"

then calling function of testmodule.testmod() print the old data as "I am in testmodule"

Then how to reflect the changes in the sample.py file.

Just you need to  use "reload(testmodule)"

----------------------------------
import testmodule   >>> Import the testmodule

testmodule.testmod()

reload(testmodule)

testmodule.testmod()


Output->
I am in testmodule
I am in sample module

====================
GETTING MODULE INFO
====================
==================================
How to get the module information
==================================

import math

dir(math) -> It gives information about the modules present
under the math.


['__doc__', 'acos', 'asin', 'fabs' ..... ]

help(math) -> Describe about each module.


The builtin function can be accessed using the "math.cos"

===================
WORKING with FILES
===================

python supports the file operations as similar to C language.


keywords: open, write, read, close.
---------

syntax for writing in to file:
------------------------------
fileObject= open('c:/test.txt', 'w')  // Here 'w' (write) or 'r' (read)

fileObject.write("Hey Muni i am here")

fileObject.close()


syntax for reading from the file:
---------------------------------

fileObject= open('c:/test.txt', 'r')  // Here 'w' (write) or 'r' (read)

fileObject.read()
fileObject.close()


Output ->
Hey Muni I am here

fileObject.read(3)

Output -> Hey
---------------------------------------
How to read and write line in files
---------------------------------------
using keyword line by line -> "readline() and writeline()"

-> To read all Lines
           - > readlines() and writelines()

--------
example:
--------

Write
-----

fobj= open('/home/test/test.txt', 'w')
fobj.write("muni sekhar reddy")
fobj.close()

write and read
--------------

fobj= open('/home/test/test.txt', 'w')
fobj.write("muni sekhar reddy")
fobj.close()


fobj= open('/home/test/test.txt', 'r')
print fobj.read()
fobj.close()

readline
--------
fobj= open('/home/test/test.txt', 'w')
fobj.write("muni sekhar reddy \n I love you \n")
fobj.close()

fobj= open('/home/test/test.txt', 'r')
print fobj.readline()
print fobj.readline()
fobj.close()


Output->

muni sekhar reddy
I love you

----------------------------------------------------
readlines   -> Reads all Lines and display as a list
----------
fobj= open('/home/test/test.txt', 'w')
fobj.write("muni sekhar reddy \n I love you \n")
fobj.close()

fobj= open('/home/test/test.txt', 'r')
print fobj.readlines()
fobj.close()

Output-> displays list.

['muni sekhar reddy \n', ' I love you \n']
-----------------------------
How to modify the file lines
-----------------------------

fobj= open('/home/test/test.txt', 'w')
fobj.write("muni sekhar reddy \n I love you \n")
fobj.close()

fobj= open('/home/test/test.txt', 'r')
listme= fobj.readlines()
fobj.close()

print listme

listme[1] = "sekhar reddy"

Now change the second line using..

fob= open('/home/test/test.txt', 'w')
fob.writelines(listme)
fob.close()

The list of lines are stored in the file. using writelines.


==============================================================================
Miscellanous and Advance concepts
==============================================================================

For loop with in a list notation  -> This is also called list comprehension.
=================================

syntax:

listme = [<variable> for i in range(10)]

This can be written as

example
-------
listme = []
listme= [ i for i in range(10,20)]
print listme


output->
[10,11,12,13,14,15,16,17,18,19]


The same can be represented as:->
-----------------------------------
listme=[]
for i in range(10,20):
 listme.append(i)
-----------------------------------

Other examples of list comprehension
--------------------

example:4
---------
data = ['3', '7.4', '8.2']
new_data = [float(n) for n in data]

Output->
[3.0, 7.4, 8.2]



example 5:
---------
myList = []
for i in range(10):
    if i%2 == 0:     # could be written as "if not i%2" more tersely
       myList.append(i)

The above condition can be comprehensed as

mylist = [i for i in range(10) if i%2==0]

Note:
-----
You can have "nested" list comrehensions, but they quickly become hard to comprehend
List comprehension will run faster than the equivalent for-loop,
and therefore is often a favorite with regular Python programmers who are concerned about efficiency


====================================================
How to start the Python program Using Main function
====================================================
--------
syntax 1:  main definition can be called as below.
--------

def main():
   print " I am in the Main function "

if __name__ == "__main__"
   main() // Call the main function here.

---------
syntax 2:
---------

def main(argv)
   print " I am in the main function with the arguments "

if __name__ == "__main__"
   main()  //Call the main function with arguments.

============================
Enumerate Keyword in Python
============================
enumerate keyword adds counter to the variable.
The enumerate() function adds a counter to an iterable.

example 1:

  for i in range(5)
    print i

Output -> 0, 1,2,3,4


example Using enumerate:
------------------------

  for count, i in enumerate(range(5))
    print count,i

Output -> count: 0,1,2,3,4  i => 0, 1,2,3,4

example Using List:
------------------------

By default enumeration starts from zero.


elementList = ['foo' , 'bar', 'baz']

for count, element in enumerate(elementlist)
  print count, element

Output-> rowindex, row value
0 foo
1 bar
2 baz

example Using List:
------------------------

elementList = ['foo' , 'bar', 'baz']

Enumeration starts from 43.


for count, element in enumerate(elementlist , 43)
  print count, element

Output->

43 foo
44 bar
45 baz


Example of List comprehension
=============================
t1=[1,2,'Hello',(1,2),999,1.23]
t2=[1,'Hello',(1,2),999]

t3=[]

for it1, e1 in enumerate(t1):
    for it2, e2 in enumerate(t2):
        if e1==e2:
            t3.append((it1,it2,e1))


(Or)

This can be optimised with list comprehension as

t3 = [(it1,it2, e1) for it1, e1 in enumerate(t1) for it2, e2 in enumerate(t2) if e1==e2]

========================
Iterables vs Generators
========================

Iterables

When you create a list, you can read its items one by one, and it’s called iteration:

>>> mylist = [1, 2, 3]
>>> for i in mylist:
...    print(i)
1
2
3

Mylist is an iterable. When you use a list comprehension, you create a list, and so an iterable:

>>> mylist = [x*x for x in range(3)]
>>> for i in mylist:
...    print(i)
0
1
4

Everything you can use “for… in…” on is an iterable: lists, strings, files…
These iterables are handy because you can read them as much as you wish,
but you store all the values in memory and it’s not always what you want when you have a lot of values.
Generators

Generators are iterators, but you can only iterate over them once.
It’s because they do not store all the values in memory, they generate the values on the fly:

>>> mygenerator = (x*x for x in range(3))
>>> for i in mygenerator:
...    print(i)
0
1
4

It is just the same except you used () instead of [].
BUT, you can not perform for i in mygenerator a second time since generators can only be used once:
they calculate 0, then forget about it and calculate 1, and end calculating 4, one by one.

=======
Yield
=======
Yield is a keyword that is used like return, except the function will return a generator.

>>> def createGenerator():
...    mylist = range(3)
...    for i in mylist:
...        yield i*i
...
>>> mygenerator = createGenerator() # create a generator
>>> print(mygenerator) # mygenerator is an object!
<generator object createGenerator at 0xb7555c34>
>>> for i in mygenerator:
...     print(i)
0
1
4

Here it’s a useless example, but it’s handy when you know your function
will return a huge set of values that you will only need to read once.

To master yield, you must understand that when you call the function,
the code you have written in the function body does not run.
The function only returns the generator object, this is a bit tricky :-)

=============================
Virtual functions in Python:
============================

By defualt all functions are virtual in python.

class parent:
  def hello(self):
     print " I am in class parent"

class child(parent):
  def hello(self):
     print " I am in child class"


def main():

  childobj = child()
  childobj.hello()

if __name__ == "__main__":
   main()


Output->

 I am in child class.


========================
Lists in the Dictionary
=========================


def main():
  list_sample = [1,2,3,4,5]

  print "the list len %d" % len(list_sample)

  dict_sample = {1: ['A', 'B'], 2:['D','E', 'F'], 3:['S', 'G', 'K']}

  dict_sample[2].append('Z')

  for items in dict_sample:
    flag=1
    for val in dict_sample[items]:
      if flag==1:
         print "if argv[%d] == val %s" % (items, val)
         flag+=1
      elif flag <len(dict_sample[items]):
           print "else if argv[%d] == val[%s]" % (items, val)
           flag+=1
      else:
           print "else  argv[%d]== %s" % (items, val)

                                                           
=============
Python Class
=============
-------------------------------------------------
Basic syntax of class : class base(object)

Constructor :  def __init__(self)

All methods should have one parameter:  self.

void display(self)
-------------------------------------------------
---------------
Class Example
---------------

class base(object):
  def __init__(self, name):
    self.name = name
    print "I am in the constructor"

  def display(self):
    print "The Name" +" " + self.name

def main():
  ob = base('muni')
  ob.display()

if __name__ == '__main__':
  main()

------------
Inheritance
------------

class base(object):
  def __init__(self, name):
    print " I am in constructor"
    self.name = name

  def display(self):
    print "I am in base class" +" " + self.name

class derived(base):
  def __init__(self, name):
    print " I am in derived"
    self.name = name

  def display(self):
    print "I am in derived class" + " "+ self.name

def main():
  ob = base("muni")
  ob.display()

  der = derived("sekhar")
  der.display()

if __name__=='__main__':
  main()


Note: Base class constructors can't be inherited in the derived class.

Output:
I am in constructor
I am in base class muni
I am in constructor
I am in derived class sekhar

-----------------------
Class Scope Variables:
-----------------------
The Variables declared outside of the methods is accessible to every object.
Same memory is shared across all the objects (Like static variable in C++)

class base(object):
 name = "Nipom"

def main():

  ob = base()
  base.name = "Reddy"
  ob1 = base()
  print ob1.name
  print ob.name
if __name__=='__main__':
  main()

Note:  The Class scope variable is modified by the "class name.<Variable> "
-----
If it is modified by the object, it will be reflected only to the object.

Output:

 Reddy
 Reddy
--------------
Example 3:
==============

class Dog:

    tricks = []             # mistaken use of a class variable

    def __init__(self, name):
        self.name = name

    def add_trick(self, trick):
        self.tricks.append(trick)

>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.add_trick('roll over')

-----------
Example 4
-----------

class Dog:

    def __init__(self, name):
        self.name = name
        self.tricks = []    # creates a new empty list for each dog

    def add_trick(self, trick):
        self.tricks.append(trick)

>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.add_trick('roll over')
>>> e.add_trick('play dead')
>>> d.tricks
['roll over']
>>> e.tricks
['play dead']
>>> e.add_trick('play dead')
>>> d.tricks                # unexpectedly shared by all dogs
['roll over', 'play dead']                                                          

Select() Call in Sockets

SELECT call is generally used  in server side (server socket ) to service the multiple clients asynchronously.  Most of the time it is been used in non blocking sockets.


select()  allow a program to monitor multiple file descriptors, waiting until one or more of the file descriptors become "ready" for some class of I/O operation (e.g., input possible). A file descriptor is considered ready if it is possible to perform the corresponding I/O operation (e.g., read(2)) without blocking.


Four macros are provided to manipulate the sets.

• FD_ZERO() clears a set.
• FD_SET() and
• FD_CLR() respectively add and remove a given file descriptor from a set.
• FD_ISSET() tests to see if a file descriptor is part of the set;

select() call also be used to timeout for the function.

#include <stdio.h>
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>

int main(void) {
    fd_set rfds;
    struct timeval tv;
    int retval;

    /* Watch stdin (fd 0) to see when it has input. */
    FD_ZERO(&rfds);
    FD_SET(0, &rfds);
 
    /* Wait up to five seconds. */
    tv.tv_sec = 5;
    tv.tv_usec = 0;

    retval = select(1, &rfds, NULL, NULL, &tv);
 
   /* Don’t rely on the value of tv now! */
    if (retval == -1)
        perror("select()");
    else if (retval)
        printf("Data is available now.\n");
        /* FD_ISSET(0, &rfds) will be true. */
    else
        printf("No data within five seconds.\n");
    return 0;
}

Output:
If there is no input from user with in 5 secs, it will print as "No data within five seconds.\n"
If there is any input in 5 secs, it will print as "Data is available now\n"
Another Example:

#include <errno.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/time.h>

int
input_timeout (int filedes, unsigned int seconds)
{
  fd_set set;
  struct timeval timeout;
  int ret;

  /* Initialize the file descriptor set. */
  FD_ZERO (&set);
  FD_SET (filedes, &set);

  /* Initialize the timeout data structure. */
  timeout.tv_sec = seconds;
  timeout.tv_usec = 0;

  /* select returns 0 if timeout, 1 if input available, -1 if error. */
  ret =  (select (FD_SETSIZE,  &set, NULL, NULL,  &timeout));

  return ret;
}

int
main (void)
{
  fprintf (stderr, "select returned %d.\n",
           input_timeout (STDIN_FILENO, 5));
  return 0;
}

Useful String Operations used in real time C language

The Following list of String operations are being used in real time C applications.

String Operations:
==================

Header File Required : #include<string.h>

------------------------
strcat(), strncat()
------------------------
Concatenate two strings into a single string.


Prototypes
------------
#include <string.h>
int strcat(const char *dest, const char *src);
int strncat(const char *dest, const char *src, size_t n);

Concatenate", for those not in the know, means to "stick together".
These functions take two strings, and stick them together, storing the result in the first string.


Return Value
-------------
Both functions return a pointer to the destination string, like most of the string-oriented functions.


Example
========

char dest[20] = "Hello";
char *src = ", World!";
char numbers[] = "12345678";

printf("dest before strcat: \"%s\"\n", dest); // "Hello"

strcat(dest, src);
printf("dest after strcat:  \"%s\"\n", dest); // "Hello, world!"

strncat(dest, numbers, 3); // strcat first 3 chars of numbers
printf("dest after strncat: \"%s\"\n", dest); // "Hello, world!123"


--------------------------------------------------------------

strchr(), strrchr()  : Find a character in a string.
---------------------------------------------------------------


strchr() : Find the first occurance of a letter in a string
----------

strrchr(): Find the last occurance of a letter in a string
---------          
(The extra "r" in strrchr() stands for "reverse"-- it looks starting at the end of the string and working backward.)

-----------
Example 1:
----------

The above function returns a pointer to the char or NULL if the letter isn't found in the string.

// "Hello, world!"
//       ^  ^  
//       A  B

char *str = "Hello, world!";
char *p;

p = strchr(str, ','); // p now points at position A
p = strrchr(str, 'o'); // p now points at position B

-----------
Example 2:
----------

// repeatedly find all occurances of the letter 'B'
char *str = "A BIG BROWN BAT BIT BEEJ";
char *p;

for(p = strchr(str, 'B'); p != NULL; p = strchr(p + 1, 'B')) {
    printf("Found a 'B' here: %s\n", p);
}

// output is:
//
// Found a 'B' here: BIG BROWN BAT BIT BEEJ
// Found a 'B' here: BROWN BAT BIT BEEJ
// Found a 'B' here: BAT BIT BEEJ
// Found a 'B' here: BIT BEEJ
// Found a 'B' here: BEEJ


-------------------------------------------------------
strstr()  : Find a string in another string.
-------------------------------------------------------
#include <string.h>
char *strstr(const char *str, const char *substr);


Return Value:
------------
You get back a pointer to the occurance of the substr inside the str, or NULL if the substring can't be found.

----------------------------------------------------------------
Example :
----------------------------------------------------------------

char *str = "The quick brown fox jumped over the lazy dogs.";
char *p;

p = strstr(str, "lazy");
printf("%s\n", p); // "lazy dogs."

// p is NULL after this, since the string "wombat" isn't in str:
p = strstr(str, "wombat");


---------------------------------------------
strtok()
---------------------------------------------
Tokenize a string.

#include <string.h>
char *strtok(char *str, const char *delim);

If you have a string that has a bunch of separators in it, and you want to break that string up into individual pieces,
this function can do it for you.

Note that it does this by actually putting a NUL terminator after the token, and then returning a pointer to the start of the token.
So the original string you pass in is destroyed, as it were.
If you need to preserve the string, be sure to pass a copy of it to strtok() so the original isn't destroyed.

Return Value:
------------
A pointer to the next token. If you're out of tokens, NULL is returned.

-------
Example
-------

// break up the string into a series of space or
// punctuation-separated words
char *str = "Where is my bacon, dude?";
char *token;

// Note that the following if-do-while construct is very very
// very very very common to see when using strtok().

// grab the first token (making sure there is a first token!)
if ((token = strtok(str, ".,?! ")) != NULL) {
    do {
        printf("Word: \"%s\"\n", token);

        // now, the while continuation condition grabs the
        // next token (by passing NULL as the first param)
        // and continues if the token's not NULL:
    } while ((token = strtok(NULL, ".,?! ")) != NULL);
}

// output is:
//
// Word: "Where"
// Word: "is"
// Word: "my"
// Word: "bacon"
// Word: "dude"
//



---------------------------------------------
strspn(), strcspn()
---------------------------------------------


#include <string.h>
size_t strspn(char *str, const char *accept);
size_t strcspn(char *str, const char *reject);

strspn() will tell you the length of a string consisting entirely of the set of characters in accept.
That is, it starts walking down str until it finds a character that is not in the set
(that is, a character that is not to be accepted), and returns the length of the string so far.

strcspn() works much the same way, except that it walks down str until it finds a character
in the reject set (that is, a character that is to be rejected.) It then returns the length of the string so far.


Return Value

The lenght of the string consisting of all characters in accept (for strspn()),
or the length of the string consisting of all characters except reject (for strcspn()


Example
========

char str1[] = "a banana";
char str2[] = "the bolivian navy on manuvers in the south pacific";

// how many letters in str1 until we reach something that's not a vowel?
n = strspn(str1, "aeiou");  // n == 1, just "a"

// how many letters in str1 until we reach something that's not a, b,
// or space?
n = strspn(str1, "ab "); // n == 4, "a ba"

// how many letters in str2 before we get a "y"?
n = strcspn(str2, "y"); // n = 16, "the bolivian nav"


-------------------

strcpy(), strncpy()   : Copy a string

---------------------

Prototypes

#include <string.h>
char *strcpy(char *dest, char *src);
char *strncpy(char *dest, char *src, size_t n);

Description

These functions copy a string from one address to another, stopping at the NUL terminator on the srcstring.

strncpy() is just like strcpy(), except only the first n characters are actually copied.
Beware that if you hit the limit, n before you get a NUL terminator on the src string,
your dest string won't be NUL-terminated. Beware! BEWARE!

(If the src string has fewer than n characters, it works just like strcpy().)

You can terminate the string yourself by sticking the '\0' in there yourself:


Example
=======

char s[10];
char foo = "My hovercraft is full of eels."; // more than 10 chars

strncpy(s, foo, 9); // only copy 9 chars into positions 0-8
s[9] = '\0';        // position 9 gets the terminator

What is Unix or Domain Socket? How to implement unix Socket in C?

In this post, i will explain about Unix/Domain Sockets.

Why Unix Sockets are Required?

You might have heard different types of sockets like TCP, UDP, SCTP, RAW etc.

These sockets are useful best for inter communications where IP address and port number (TCP /UDP Header) is most important.

When there are two process let's say X and Y reside in the same system(machine) needs to communicate each other, then which sockets would be the better option?

For the above case, i would prefer to use UNIX sockets rather than TCP/UDP sockets, since there is no over head of TCP/UDP header.  Unix Sockets doesn't carry any TCP/UDP header for communication.

What is Unix Socket?

A Unix domain socket  is a data communications endpoint for exchanging data between processes executing within the same host operating system. 

While similar in functionality to named pipes.Unix domain sockets may be created as connection‑mode (SOCK_STREAM or SOCK_SEQPACKET) or as connectionless (SOCK_DGRAM), while pipes are streams only. 

Processes using Unix domain sockets do not need to share a common ancestry. The API for Unix domain sockets is similar to that of an Internet socket, but it does not use an underlying network protocol for communication. 

Unix domain sockets use the file system as their address name space. They are referenced by processes as inodes in the file system. This allows two processes to open the same socket in order to communicate. However, communication occurs entirely within the operating system kernel.

In addition to sending data, processes may send file descriptors across a Unix domain socket connection using the sendmsg() and recvmsg() system calls. 

How to Implement Unix Sockets?

In Unix sockets,  the socket API are similar to internet socket.

Unix Socket create using API:

 if ((sock_fd = socket(AF_UNIX, SOCK_STREAM, 0)) < 0) {
        perror("server: socket");
        return -1;
    }

    /*
     * Create the address we will be connecting to.
     */
    address.sun_family = AF_UNIX;
    snprintf(address.sun_path, UNIX_PATH_MAX, "demo_socket");

UNIX CLIENT PROGRAM

-----------------------------------

#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <stdio.h>

#define UNIX_PATH_MAX 108

/*
 * Strings we send to the server.
 */
char str[100] ;

int main()
{
    int sock_fd= -1, i, len;
 
    /* Unix Sock Address structure */
    struct sockaddr_un address;

    /*
     * Create domain socket.
     * This socket will be in the UNIX domain,
     * and will be a stream socket.
     */
    if ((sock_fd = socket(AF_UNIX, SOCK_STREAM, 0)) < 0) {
        perror("server: socket");
        return -1;
    }
     /* start with a clean address structure */
    memset(&address, 0, sizeof(struct sockaddr_un));

    /*
     * Create the address we will be connecting to.
     */
    address.sun_family = AF_UNIX;
    snprintf(address.sun_path, UNIX_PATH_MAX, "demo_socket");


    /*
     * Connect to the address.  For this to
     * succeed, the server must already have bound
     * this address, and must have issued a listen()
     * request.
     *
     * The third argument indicates the "length" of
     * the structure.
     *
     */
    if(connect(sock_fd,
            (struct sockaddr *) &address,
            sizeof(struct sockaddr_un)) != 0)
     {
       printf("connect() failed\n");
       return 1;
     }

    /* Send and recieve Data
     *   in while Loop
     */
    while(printf("> "), fgets(str, 100, stdin), !feof(stdin))
    {
        /* Send the Data to server */
        if (send(sock_fd, str, strlen(str), 0) == -1)
         {
            perror("send");
            return 1;
         }

        /* Recieve the Date from Server */
        if ((len=recv(sock_fd, str, 100, 0)) > 0)
         {
            str[len] = '\0';
            printf("echo> %s", str);

         }
        else
         {
            if (len < 0)
              perror("recv");
            else // Length = 0
              printf("Server closed connection\n");
              return 1;
        }
     }
    /*
     * We can simply use close() to terminate the
     * connection, since we're done with both sides.
     */
    close(sock_fd);

    return 0;
}

UNIX SERVER PROGRAM

-----------------------------------

#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <stdio.h>

#define UNIX_PATH_MAX  108

char str[100];

int main()
{

    /* Sock Fd and Connection Fd = -1 */
    int  sock_fd= -1, connection_fd = -1, listen_fd =-1, i;

   /* unix Socket address Structure */
    struct sockaddr_un address;
   
    /* Socket Address Len */
    socklen_t address_len;

    /*
     * Get a socket to work with.  This socket will
     * be in the UNIX domain, and will be a
     * stream socket.
     */
    if ((sock_fd = socket(AF_UNIX, SOCK_STREAM, 0)) < 0) {
        perror("server: socket");
        return -1;
    }

     /* start with a clean address structure */
     memset(&address, 0, sizeof(struct sockaddr_un));

    /*
     * Create the address we will be binding to.
     */
     address.sun_family = AF_UNIX;
     snprintf(address.sun_path, UNIX_PATH_MAX, "demo_socket");

    /*
     * Bind the address to the socket.  We
     * unlink the name first so that the bind won't
     * fail.
     *
     * The third argument indicates the "length" of
     * the structure, not just the length of the
     * socket name.
     */

     unlink("demo_socket");
    
     if(bind(sock_fd,
                 (struct sockaddr *) &address,
                 sizeof(struct sockaddr_un)) != 0)
      {
          printf("bind() failed\n");
          return 1;
      }

    /*
     * Listen on the socket.
     */
      if( listen_fd = listen(sock_fd, 5) != 0)
       {
          printf("listen() failed\n");
          return 1;
       }
    
    /*
     * Accept connections.  When we accept one, ns
     * will be connected to the client.  fsaun will
     * contain the address of the client.
     */
    while (1)
    {
       int done, len;

       connection_fd = accept(sock_fd,
             (struct sockaddr *)&address,
             &address_len);
       if(connection_fd < 0)
       {
         perror("server: accept");
         return -1;
       }
      
       close(listen_fd);
      
       /* Recv and Send in Infinite Loop */
       done = 0;
       do
       {
          /* Recieve Data from Client */
          len = recv(connection_fd, str, 100, 0);

          if (len <= 0)
          {
             if (len < 0)
                perror("recv");
             done = 1;
          }

          if (!done)
             /* Send Data to Client */
             if (send(connection_fd, str, len, 0) < 0)
              {
                  perror("send");
                  done = 1;
              }
        } while (!done);

      /*
       * We can simply use close() to terminate the
       * connection, since we're done with both sides.
       */
   
      close(connection_fd);
    }
    return 0;
}

--------------------------------------------------------------
 HOW UNIX SOCKETS ARE USED IN REAL TIME APPLICATIONS
------------------------------------------------------------- 
Explain With simple example:

Let's take two process x_process (as a client) and y_process(as a server.
x_process have different call back functions and it shall be triggered based on the message type send from the server(y_process).


x_process registers the call back functions for different Message types during initialization time.

Message Types : HELLO_MESSAGE, DOWN_MESSAGE, UPDATE_MESSAGE.

Based on the respective message, the corresponding call back function is called.

x_process   (CLIENT)
------------
#include<stdio.h>
#include<stdlib.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>

#include "message.h"

#define SOCK_PATH "echo_socket"


/* Hello Message Handle Function */
int handle_hello_message_func(int message_type)
{
   printf(" I am in Hello function \n");
   return 0;
}

/* Down Message Handle Function */
int handle_down_message_func(int message_type)
{
   printf(" I am in down function \n");
   return 0;
}

/* Update Message Handle Function */
int handle_update_message_func(int message_type)
{
   printf(" I am in update function \n");
   return 0;
}

/*
 * Trigger Callback function
 */

void trigger_callback_function(char *str, struct handle_msg *msg)
{

  printf("%s", str);

  if(strcmp(str, "HELLO_MESSAGE\n") == 0)
      msg->callback_function[HELLO_MESSAGE](HELLO_MESSAGE);
  else if (strcmp(str, "UPDATE_MESSAGE\n") == 0)
      msg->callback_function[UPDATE_MESSAGE](UPDATE_MESSAGE);
  else if (strcmp(str, "DOWN_MESSAGE\n") == 0)
      msg->callback_function[DOWN_MESSAGE](DOWN_MESSAGE);
  else
      printf(" Message is not Known, Ignored ");
}

/*
 * Unix Socket Creation
 *
 */ 

int unix_socket_init(struct handle_msg **msg)
{
   int len, recv_len, t,s;
 
   /* Declare the remote (Server) structure */
   struct sockaddr_un remote;

   char str[100];

   /* Create Unix Socket */
   (*msg)->sock_fd = socket(AF_UNIX, SOCK_STREAM, 0);

   if((*msg)->sock_fd < 0)
    {
      perror("socket");
      return 1;
    }
  
   printf("Trying to connect...\n");

   /* Set the server Attributes */
   remote.sun_family = AF_UNIX;
   strcpy(remote.sun_path, SOCK_PATH);
   
   /* Calculate the length */
   len = strlen(remote.sun_path) + sizeof(remote.sun_family);
  
   /* Connect to the server */
   if (connect((*msg)->sock_fd, (struct sockaddr *)&remote, len) == -1) {
        perror("connect");
        return 1;
    }

   printf("Connected.\n");

   s= (*msg)->sock_fd;

   /* Send and recieve in while Loop */
   while(printf("> "), fgets(str, 100, stdin), !feof(stdin)) {
        if (send(s, str, strlen(str), 0) == -1) {
            perror("send");
            exit(1);
        }

        if ((t=recv(s, str, 100, 0)) > 0) {
            str[t] = '\0';
            printf("echo> %s", str);

        /* Trigger the Respective Call back Function */
        trigger_callback_function(str, *msg); 

         
        } else {
            if (t < 0) perror("recv");
            else printf("Server closed connection\n");
            exit(1);
        }
     }
   close(s);
}

/* Intialise the X process
 * Step 1 : Create Unix Socket
 * Step 2 : Register the CallBack functions with the Lib Code
 *
 *
 */
void x_client_process_init()
{

   struct handle_msg *msg= NULL;

   /* Allocate Message Structure */
   msg= (struct handle_msg *)malloc(sizeof(struct handle_msg));


   /* Register the server HELLO call back function */
   client_register_callback_function(msg, HELLO_MESSAGE,
                                     handle_hello_message_func );

   /* Register the server DOWN call back function */
   client_register_callback_function(msg, DOWN_MESSAGE,
                     handle_down_message_func);

   /* Register the server DOWN call back function */
   client_register_callback_function(msg, UPDATE_MESSAGE,
                     handle_update_message_func);
 
   /* Create Unix Socket and  connect to the server*/
   unix_socket_init(&msg);
}

int main()
{
   /* Intialise the X Client Process */
   x_client_process_init();
  
 
   return 0;
}
message.h  (Common File)
========================

/* DECLARE THE DIFFERENT Message TYPES between Server and Client */
#define HELLO_MESSAGE                   0
#define UPDATE_MESSAGE                  1
#define DOWN_MESSAGE                    2
#define MAX_MESSAGE                     3

/* There are many ways to declare array of Call back functions */
/* One way is :
 *  -> int (*fun[10])(int, int) or
 *  -> typedef int (*func)(int, int); func arrayfunc[10]
 */
/* Declare the Message Call back handler function */
typedef int (*CALLBACK_MESSAGE)(int message_type);




/* Common Message Structure for Client and Server */
struct handle_msg
{

  /* Socket to accept or connect.  */
  int sock_fd;

  /* functions registers in to this call back function */
  CALLBACK_MESSAGE callback_function[MAX_MESSAGE];
};

/* Register the call back functions
 * once the functions are registered and will be
 * triggered dynamically
 *
 */

void client_register_callback_function(struct handle_msg *msg,
                              int message_type,
                              CALLBACK_MESSAGE cb)
{
  if (message_type >= MAX_MESSAGE)
    return;

  msg->callback_function[message_type] = cb;
}

Y_PROCESS (Server)
================
#include<stdio.h>
#include<stdlib.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>

#include "message.h"

#define SOCK_PATH "echo_socket"

/*
 * Server Unix Socket Creation
 *
 */
int server_unix_socket_init(struct handle_msg **msg)
{

   int s, connect_fd, t, len;
   struct sockaddr_un local, remote;
   char str[100];


   /* Create Unix Socket */
   (*msg)->sock_fd = socket(AF_UNIX, SOCK_STREAM, 0);

   if((*msg)->sock_fd < 0)
    {
       perror("socket");
       return 1;
    }

   /* Populate the local structure */
   local.sun_family = AF_UNIX;
   strcpy(local.sun_path, SOCK_PATH);
  
   /* Unlink the path */
   unlink(local.sun_path);
  
   len = strlen(local.sun_path) + sizeof(local.sun_family);
  
   /*
    * Bind to the Local path -> For Unix Sockets there will not
    * be any port Num or IP address
    *
    * It will bind to the local path.
    */   

   if (bind((*msg)->sock_fd, (struct sockaddr *)&local, len) == -1)
    {
        perror("bind");
        return 1;
    }

    if (listen((*msg)->sock_fd, 5) == -1) {
        perror("listen");
        return 1;
    }

    for(;;)
      {
        int done, n;
        printf("Waiting for a connection...\n");
        len = sizeof(remote);
       
  
        connect_fd = accept((*msg)->sock_fd, (struct sockaddr *)&remote, &len);
       
        if((*msg)->sock_fd < 0)
         {
            perror("accept");
            return 1;
         }

        printf("Connected.\n");

 
        done = 0;
        do {
            n = recv(connect_fd, str, 100, 0);
            if (n <= 0) {
                if (n < 0) perror("recv");
                done = 1;
            }

            if (!done)
                if (send(connect_fd, str, n, 0) < 0) {
                    perror("send");
                    done = 1;
                }
        } while (!done);
         
        /* Send and recieve in while Loop */
        close(connect_fd);
     }
}
/*
 * Y server process init()
 */
void y_server_process_init()
{
   struct handle_msg *msg= NULL;

   /* Allocate Message Structure */
   msg= (struct handle_msg *)malloc(sizeof(struct handle_msg));


   /* Create Server Unix Socket, listen and accept the connection*/
   server_unix_socket_init(&msg);

}
/*
 * Main()
 */
int main()
{
   /* Intialise the Y Process */
   y_server_process_init();

}


NOTE: Give Input as " HELLO_MESSAGE\n"  or DOWN_MESSAGE\n, UPDATE_MESSAGE\n to get proper output.   "\n" is getting added to the string while reading.

What is SKB in Linux kernel? What are SKB operations? Memory Representation of SKB? How to send packet out using skb operations?

As many of them are aware, OSI reference and TCP/IP Model.

For any Networking application TCP/IP model is required to process/route the packets from one end to other. Then, How to support the networking in Linux kernel?  How to process the packets in Linux kernel?   

This article will answer the above questions.

•   A network interface represents a thing which sends and receives packets. This is normally interface code for a physical device like an ethernet card. However some devices are software only such as the loopback device which is used for sending data to yourself.
• The network subsystem of the Linux kernel is designed to be completely protocol-independent. This applies to both networking protocols (Internet protocol [IP] versus IPX or other protocols) and hardware protocols (Ethernet versus token ring, etc.).
•    A header is a set of bytes (err, octets) prepended to a packet as it is passed through the various layers of the networking subsystem. When an application sends a block of data through a TCP socket, the networking subsystem breaks that data up into packets and puts a TCP header, describing where each packet fits within the stream, at the beginning. The lower levels then put an IP header, used to route the packet to its destination, in front of the TCP header. If the packet moves over an Ethernet-like medium, an Ethernet header, interpreted by the hardware, goes in front of the rest.

The two important data structures of Linux kernel network layer are:

-         sk_buff     (defined  in  /include/linux/sk_buff.h)

-         net_device  (defined  in  /include/linux/net_device.h) Each interface is described by a struct net_device item. ( Please refer in another post)

What is sk_buff:

• sk_buff means socket buffers. This is core structure in Linux networking.
•  skbuffs are the buffers in which the Linux kernel handles network packets. The packet is received by the network card, put into a skbuff and then passed to the network stack, which uses the skbuff all the time.
• In the same words as above:

As we need to manipulate packets through the Linux kernel stack, this manipulation involves efficiently:  

• Adding protocol headers/trailers down the stack.
•  Removing protocol headers/trailers up the stack. 
•  Concatenating/separating data. 
• Each protocol should have convenient access to header fields.

To order to perform all of above, kernel provides the sk_buff structure.

Structure of sk_buff

sk_buff structure is created when an application passes data to a socket or when a packet arrives at the network adaptor (dev_alloc_skb() is invoked).

MEMORY Representation of SKB structure:

SKB has four parts. Memory representation of skb structure is depicted below.

sk_buff has five pointers as mentioned below.

head	the start of the packet
data	the start of the packet payload
tail	the end of the packet payload
end	the end of the packet
len	the amount of data of the packet

As shown in above figure, Skb memory usually has four parts:

1.       head room : located skb-> between the head and skb-> data, which is stored in the local network protocol header, such as TCP, IP header, Ethernet header are located here;

2.       User data  : usually filled by the application layer calls through the system between skb-> data and skb-> tail between;

3.       tail room : between skb-> tail and skb-> end, which is the core part of the user data to fill in the back part;

4.       skb-> after the end is stored in a special structure struct skb_shared_info.

STEPS for sending the packet out using SKB OPERATIONS:

Step 1:   Allocate Memory for the skb

skb = alloc_skb(len, GFP_KERNEL);

Once the skb is allocated with memory using alloc_skb() then it will look like as shown in below.

As you can see, the head, data, and tail pointers all point to the beginning of the data buffer. And the end pointer points to the end of it. Note that all of the data area is considered tail room.

The length of this SKB is zero, it isn't very interesting since it doesn't contain any packet data at all. 

Step 2:  Space for Headroom to add protocol headers (Ethernet + IP+ TCP headers etc..)

Reserve some space for protocol headers using skb_reserve().It usually initialize the head room, by calling skb_reserve () function as shown in figure below.

skb_reserve(skb, header_len);

skb->data and skb->tail pointer increments (advances or moves) by the specified Header length.

For example, the TCP layer to send a data packet, head room, at least if tcphdr + iphdr + ethhdr. 

Step 3 :  Add the User Data (payload) after the skb->put()

skb_put() advances (moves the) 'skb->tail' pointer by the specified number of bytes (user_data_len), it also increments 'skb->len' by that number of bytes as well. This routine must not be called on a SKB that has any paged data. 

unsigned char *data = skb_put(skb, user_data_len);                    

memcpy(data, 0x11,  user_data_len);

Step 3:  Add the Headers Using the skb->push()  in the Headroom


skb_push() will decrement the 'skb->data' pointer by the specified number of bytes. It will also increment'skb->len' by that number of bytes as well.  Make sure that there is enough head room for the push being performed.
For example , push the TCP header to the front of the SKB.


unsigned char *tcp_header = skb->push(skb, sizeof(struct udphdr));
struct tcphdr *tcp;
tcp = tcp_header;
 
tcp->source = htons(1025);
tcp->dest = htons(2095);
tcp->len = htons(user_data_len);
tcp->check = 0;
 

skb->pull()   :

Remove the respective headers From the Headroom and returning the bytes to headroom using skb_pull() operation.

It Increments (pulled down) the skb-> data by a specified number of bytes and decrements the skb_len.

Step 5: Similarly Push the Ipv4 header in to sk_buffer using

skb_push operation and send the packet out using dev_queue_xmit() function.

Please refer the example code below.

Pictorial representation of skb operations.

Some more skb operations 

skb_clone(),skb_copy(),skb_trim()

skb_clone() : This function will not copy the entire skb structure. It is just make the skb to point to the same region of memory on the line.

In this case data_ref pointer present in the skb_shared_info structure will be incremented to 2

For example:

Packets can be captured using "tcpdump" in linux kernel. At this point, the packet will be given to linux stack and the same packet is given to tcpdump as well (First protocol stack and after tcpdump). 

In this case not to completely copy skb it? Not necessary, because the two parts are read, the network data itself is unchanged, becomes just strcut sk_buff pointer inside, that is to say, we just copy skb to point to the same region of memory on the line! This is skb_clone () did:

It can also access using by skb + 1

skb_copy()
In some cases we need have to write a complete copy of their data skb call skb_copy().It copies entire data of skb and creates another skb. 

skb_trim()  : remove end from a buffer

Prototype

void skb_trim(struct sk_buff *skb, unsigned int len);

skb support functions

There are a bunch of skb support functions provided by the sk_buff layer. 

allocation / free / copy / clone and expansion functions

struct sk_buff *alloc_skb(unsigned int size, int gfp_mask)

This function allocates a new skb. This is provided by the skb layer to initialize some privat data and do memory statistics. The returned buffer has no headroom and a tailroom of /size/ bytes.

void kfree_skb(struct sk_buff *skb)

Decrement the skb's usage count by one and free the skb if no references left.

struct sk_buff *skb_get(struct sk_buff *skb)

Increments the skb's usage count by one and returns a pointer to it.

struct sk_buff *skb_clone(struct sk_buff *skb, int gfp_mask)

This function clones a skb. Both copies share the packet data but have their own struct sk_buff. The new copy is not owned by any socket, reference count is 1.

struct sk_buff *skb_copy(const struct sk_buff *skb, int gfp_mask)

Makes a real copy of the skb, including packet data. This is needed, if You wish to modify the packet data. Reference count of the new skb is 1.

struct skb_copy_expand(const struct sk_buff *skb, int new_headroom, int new_tailroom, int gfp_mask)

Make a copy of the skb, including packet data. Additionally the new skb has a haedroom of /new_headroom/ bytes size and a tailroom of /new_tailroom/ bytes.

anciliary functions

int skb_cloned(struct sk_buff *skb)

Is the skb a clone?

int skb_shared(struct sk_Buff *skb)

Is this skb shared? (is the reference count > 1)?

operations on lists of skb's

struct sk_buff *skb_peek(struct sk_buff_head *list_)

peek a skb from front of the list; does not remove skb from the list

struct sk_buff *skb_peek_tail(struct sk_buff_head *list_)

peek a skb from tail of the list; does not remove sk from the list

__u32 skb_queue_len(sk_buff_head *list_)

return the length of the given skb list

void skb_queue_head(struct sk_buff_head *list_, struct sk_buff *newsk)

enqueue a skb at the head of a given list

void skb_queue_tail(struct sk_buff_head *list_, struct sk_buff *newsk)

enqueue a skb at the end of a given list. 

int skb_headroom(struct sk_buff *skb)

return the amount of bytes of free space at the head of skb

int skb_tailroom(struct sk_buff *skb)

return the amount of bytes of free space at the end of skb

struct sk_buff *skb_cow(struct sk_buff *skb, int headroom)

if the buffer passed lacks sufficient headroom or is a clone it is copied and additional headroom made available. 

void struct sk_buff *skb_dequeue(struct sk_buff_head *list_)

            skb_dequeue() takes the first buffer from a list (dequeue a skb from the head of the given list) If the list is empty a NULL pointer is returned. This is used to pull buffers off queues. The buffers are added with the routines skb_queue_head() andskb_queue_tail().

struct sk_buff *sbk_dequeue_tail(struct sk_buff_head *list_)

              dequeue a skb from the tail of the given list 

Network device packet flow:

How to Identify skb is linear or not.
Data is placed between skb-> head and skb-> end, it is called as linear (linear).
skb->data_len == 0  is Linear.

else

If skb is not linear  means  skb->data_len != 0, 

the length of  skb->data is (skb->len) - (skb-> data_len) for the head ONLY.

Pseudocode
----------

/* SKB Is Linear */

  if (skb->data_len == 0)

    {

      printk(" Skb is Linear : the skb_len is :%d", skb->len);

    }

  /* SKB Is Not Linear */

  else

    {

      if(skb->data_len != 0)

        {

          skb->len = (skb->len) - (skb->data_len);

        }

       Printk(“Skb is Not Linear : the skb_len is :%d", skb->len);

    }

skb->data_len =   struct skb_shared_info->frags[0...struct skb_shared_info->nr_frags].size

                                          + size of data in struct skb_shared_info->frag_list

When SKB is Non Linear?

First Model :

One is the common NIC driver model, data is stored in different locations of physical pages, skb_shared_info there are an array to store a set of (page, offset, size) of the information used to record these data

Second Model:

frag_list assembled IP packet fragmentation (fragment) when used in: 

Fragmentation of data has its own skb structure, which through skb-> next link into a single linked list, the first table is the first one in the skb shared_info frag_list.

Third  Model:

GSO segment (segmentation) used a model, when a large TCP data packets are cut into several MTU size, they are also through skb-> next link together:

References:

http://wangcong.org/blog/archives/2337

http://www.skbuff.net/skbbasic.html

http://vger.kernel.org/~davem/skb_data.html

http://blog.csdn.net/npy_lp/article/details/7263902