Python : Converting unicode logs of dict into valid json format

While debugging, programmer goes through logs to grab request payload. The issue is payload is wrapped with u for unicode, ‘ and True and False and None which are python standards however fails to pass json validation.

Hence to use them, we need to convert the unicoded python dict into valid json to do crul call, for the same here is the simple trick :

{u’a’: u’aValue’, u’b’: u’bValue’, u’c’: u’cValue’} is a dictionary which you are calling as unicode json. Now, in your language if you want a regular json from this then just do something like this:

x={u'a': u'aValue', u'b': u'bValue', u'c': u'cValue'}
y=json.dumps(x)
print y

The output will be {“a”: “aValue”, “c”: “cValue”, “b”: “bValue”}

This will remove all u, convert False into false, True into true, None into null and convert ‘ into ”   🙂

Python : What is Virtual Environment

Each project might need some set of dependencies with their specific version to work properly. Usually, we might have multiple projects running on different versions / packages. For example

  • some project on python2 others python 3
  • some project using pymongo2.7 others pymongo3.2

Normally, we have installed python 2.7, then we might not run python3 without upgrading to python3, however there is solution called virtual Environment which allows us to have an environment for specific required for each project and it is called as virtual environment. This can be installed using below command

$ [sudo] pip install virtualenv

The trick is to create separate folder for each project and activate virtual environment over there, that way requirements might not clash with each other. 

for example, i have 2 projects named make_money and spend_money wh

So I have to make 2 different directories and install virtual environment in them, I can run my project from any where, however the dependencies would be installed in that specific project. 

It is kind of similar to NPM for node, and pip is similar to maven in java

Helpful link : https://www.dabapps.com/blog/introduction-to-pip-and-virtualenv-python/

Python : Profiling / timing your code

A good code not only gives expected results but also performs better (i.e takes less time ).

Sometimes its important to know how long your code is taking to run, or at least know if a particular line of code is slowing down your entire project. Python has a built-in timing module to do this.

This module provides a simple way to time small bits of Python code. It has both a Command-Line Interface as well as a callable one. It avoids a number of common traps for measuring execution times.

Lets learn about timeit!

We’ll pass two arguments the actual line we want to test encapsulated as a string and the number of times we wish to run it. Here we’ll choose 10,000 runs to get some high enough numbers to comapre various methods.

import timeit

# For loop
time_to_execute = timeit.timeit(‘”-“.join(str(n) for n in range(100))’, number=10000)
print time_to_execute

# List comprehension
time_to_execute = timeit.timeit(‘”-“.join([str(n) for n in range(100)])’, number=10000)
print time_to_execute

# Map()
time_to_execute = timeit.timeit(‘”-“.join(map(str, range(100)))’, number=10000)
print time_to_execute

 

$$$$$  Output $$$$$$$$$
/usr/bin/python /Users/asee2278/gitRepo/petProjects/pythonPlayground/oop/Debugger.py

0.225734949112
0.197602033615
0.122097969055

We observed that all blocks of code were doing exact same thing, but map() wins over other in terms of performance.

Python : Debugging using Python’s built-in debugger module (pdb)

You’ve probably used a variety of print statements to try to find errors in your code. A better way of doing this is by using Python’s built-in debugger module (pdb). The pdb module implements an interactive debugging environment for Python programs. It includes features to let you pause your program, look at the values of variables, and watch program execution step-by-step, so you can understand what your program actually does and find bugs in the logic.

Run below example code and observer that program stops at break point where “pdb.set_trace()” is writter, you can watch value of all variable like x,y,z after that point.

 

import pdb

x = [1,3,4]
y = 2
z = 3

result = y + z
print result

# Set a trace using Python Debugger
pdb.set_trace()

result2 = y+x
print result2

Now we could check what the various variables were and check for errors. You can use ‘q’ to quit the debugger. For more information on general debugging techniques and more methods, check out the official documentation: https://docs.python.org/2/library/pdb.html

Python : What is tuple in Python

Basic definition : Tuple is a read only list. Its elements cannot be re assigned nor its size can be altered / changed (as we do in lists), and all other operation like slicing, iteration, concatenation can be done in a similar way as lists.

 

A tuple is a sequence of immutable Python objects. Tuples are sequences, just like lists. The differences between tuples and lists are, the tuples cannot be changed unlike lists and tuples use parentheses, whereas lists use square brackets.

Creating a tuple is as simple as putting different comma-separated values. Optionally you can put these comma-separated values between parentheses also. For example −

tup1 = ('physics', 'chemistry', 1997, 2000);
tup2 = (1, 2, 3, 4, 5 );
tup3 = "a", "b", "c", "d";

The empty tuple is written as two parentheses containing nothing −

tup1 = ();

To write a tuple containing a single value you have to include a comma, even though there is only one value −

tup1 = (50,);

Like string indices, tuple indices start at 0, and they can be sliced, concatenated, and so on.

Accessing Values in Tuples:

To access values in tuple, use the square brackets for slicing along with the index or indices to obtain value available at that index. For example −

#!/usr/bin/python

tup1 = ('physics', 'chemistry', 1997, 2000);
tup2 = (1, 2, 3, 4, 5, 6, 7 );

print "tup1[0]: ", tup1[0]
print "tup2[1:5]: ", tup2[1:5]

When the above code is executed, it produces the following result −

tup1[0]:  physics
tup2[1:5]:  [2, 3, 4, 5]

Updating Tuples

Tuples are immutable which means you cannot update or change the values of tuple elements. You are able to take portions of existing tuples to create new tuples as the following example demonstrates −

#!/usr/bin/python

tup1 = (12, 34.56);
tup2 = ('abc', 'xyz');

# Following action is not valid for tuples
# tup1[0] = 100;

# So let's create a new tuple as follows
tup3 = tup1 + tup2;
print tup3

When the above code is executed, it produces the following result −

(12, 34.56, 'abc', 'xyz')

Delete Tuple Elements

Removing individual tuple elements is not possible. There is, of course, nothing wrong with putting together another tuple with the undesired elements discarded.

To explicitly remove an entire tuple, just use the del statement. For example:

#!/usr/bin/python

tup = ('physics', 'chemistry', 1997, 2000);

print tup
del tup;
print "After deleting tup : "
print tup

This produces the following result. Note an exception raised, this is because after del tup tuple does not exist any more −

('physics', 'chemistry', 1997, 2000)
After deleting tup :
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    print tup;
NameError: name 'tup' is not defined

Basic Tuples Operations

Tuples respond to the + and * operators much like strings; they mean concatenation and repetition here too, except that the result is a new tuple, not a string.

In fact, tuples respond to all of the general sequence operations we used on strings in the prior chapter −

Python Expression Results Description
len((1, 2, 3)) 3 Length
(1, 2, 3) + (4, 5, 6) (1, 2, 3, 4, 5, 6) Concatenation
(‘Hi!’,) * 4 (‘Hi!’, ‘Hi!’, ‘Hi!’, ‘Hi!’) Repetition
3 in (1, 2, 3) True Membership
for x in (1, 2, 3): print x, 1 2 3 Iteration

Indexing, Slicing, and Matrixes

Because tuples are sequences, indexing and slicing work the same way for tuples as they do for strings. Assuming following input −

L = ('spam', 'Spam', 'SPAM!')

 

Python Expression Results Description
L[2] ‘SPAM!’ Offsets start at zero
L[-2] ‘Spam’ Negative: count from the right
L[1:] [‘Spam’, ‘SPAM!’] Slicing fetches sections

Note : Tuple elements cannot be re assigned, however entire tuple can be re assigned.

tup = (7,8,6)

tup[0] = 99  #not allowed

tup = (1,2,3) # re assigned is allowed

Python : What is the use of continue statement

The continue statement uses to skip the code in for iteration of for loop and continue with next round. In below example if you get an even number you want to print and wants to take control to start of for loop and skip remaining part, use continue.

>>> for num in range(2, 10):
...     if num % 2 == 0:
...         print "Found an even number", num
...         continue
...     print "Found a number", num
Found an even number 2
Found a number 3
Found an even number 4
Found a number 5
Found an even number 6
Found a number 7
Found an even number 8
Found a number 9