In python, objects are referred as references

Python uses shallow copy for object copying. If you see this code, The value b.a.val is modified externally.

class A(object):
    def __init__(self):
        self.val = 20
    
class B(object):
    def __init__(self, a):
        self.a = a
    
if __name__ == "__main__":
    a = A()

    b = B(a)
    print b.a.val # 20
    a.val = 40
    print b.a.val # 40

You need to use copy.deepcopy to avoid this pitfall.

from copy import *

class A(object):
    def __init__(self):
        self.val = 20
    
class B(object):
    def __init__(self, a):
        self.a = deepcopy(a)
        
    
if __name__ == "__main__":
    a = A()

    b = B(a)
    
    print b.a.val
    a.val = 40
    print b.a.val

doctest and then run sphinx

When you need to run unit-test and make documentation for it with Python. doctest and sphinx is the one of the best choices you can have.

"""
This is the "example" module.

The example module supplies one function, factorial().  For example,

>>> factorial(5)
120

"""

def factorial(n):
    """Return the factorial of n, an exact integer >= 0.

    If the result is small enough to fit in an int, return an int.
    Else return a long.

    >>> [factorial(n) for n in range(6)]
    [1, 1, 2, 6, 24, 120]
    """
    if n == 0:
        return 1
    else:
        return n * factorial(n-1)
        
if __name__ == "__main__":
    import doctest
    doctest.testmod()

Then execute `sphinx-quickstart`, modify `conf.py` to make Python importable the source code directory. Finally, run make html.

Directives in reST

In reST, directive is a general block of explicit markup. It is widely used extension method for reST. You can think of it a method call with a parameter. For example, the image directive is a function that has an input as path to the picture; it returns an output of HTML img tag.

> .. image:: ./image/img.png

> <img alt="./image/img.png" src="./image/img.png" />

reST also enables to incorporate img tag wherever you want with |…| attached in front of the image directive.

|bio| is bio

.. |bio| image:: ./image/img.png

<p><img alt="bio" src="./image/img.png" /> is bio

Maybe, one the most popular directives is code block in Sphinx. This is an example of Python code block, but you can use other languages such as Ruby, C, and even reST itself.

.. code_block:: python
  
   python code ...

You have python code, how to document your API with Sphinx?

Simple four steps to get the HTML files from your python code

You just need four steps.

• Execute sphinx_quickstart, and make sure that you check autodoc mode on:> autodoc: automatically insert docstrings from modules (y/N) [n]: y
• In conf.py, add the path to your source code: sys.insert('./../src')
• Execute sphinx-apidoc -o . ../src/
• Execute make html

Now, you will have your python code in HTML format.

Include reST documents into index

Topic and comments

topic directive creates

tag in HTML. It seems to be pretty useful in describing what the reST document does.

.. topic:: What is Sphinx and the RST syntax ?
 
   CONTENTS ...

When you just use two dots, it works as a comment.

toctree

When you open the index.html, you will see nothing in the contents. It is because there is nothing under the toctree directive. Open the index.rst file which is the main reST for your project. You need to add the modules.

.. toctree::
   :maxdepth: 2
   
   hello1

You may not want to show toctree. This will notify Sphinx of the document hierarchy, but skip inserting links into the document.

.. toctree::
   :hidden:

You can make toctree numbered with :numbered: field.

.. toctree::
    :maxdepth: 1
    :numbered:

:doc:`…`

In this case, you can explicitly include the reST document with :doc:`…` field.

:doc:`hello1`
  Instructions on how to get the distribution.

API documentation

In step3, we can get (semi) automatically generated API documentation file from Sphinx with sphinx-apidoc command. Whenever you want to add more documentation to it, you have to do it manually.

Including math in the documentation

Add 'sphinx.ext.pngmath' in the extensions list. Then, use math directive or field:

:math:`a^2 + b^2 = c^2`.

.. math::

(a + b)^2 = a^2 + 2ab + b^2
(a - b)^2 = a^2 - 2ab + b^2

You need to specify the latex location in your computer. sphinx-build -b html -D pngmath_latex=`which latex` . _build/html.

Useful Sphinx knowledge

Extensions

In conf.py, you can add more Sphinx features with extensions list.

extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.viewcode','sphinx.ext.todo']

References

• sphinx.ext.autodoc – Include documentation from docstrings
• TOC tree in Sphinx
• Documenting your project Using Sphinx
• sample doc tutorial

Type checking in Python

Python's value has a type, however a variable can be assigned to any typed value.

>>> a = A()
>>> a is A
False
>>> id(a)
3567120
>>> id(A)
2377856

>>> id(type(a))
2377856

>>> type(a) is A
True

>>> a.__class__ is A
True

>>> isinstance(a, A)
True

Equalities

It seems to be simple to compare two values: 1 == 1 or 2 == 3? However, when a variable is assigned to a value, it's a little bit more complicated: x = 3, y = 3, x == y? The different type of data can represent the same value: 1.0 == 0?

Lisp

> (= 1.0 1)
T

> (eql 1.0 1)
NIL

> (eql (cons 'a nil) (cons 'a nil)
NIL

> (setf x (cons 'a nil))
> (eql x x)
T

> (equal x (cons 'a nil))
T

C/C++

int x = 10;
int y = 20;

x == y // equal in Lisp
&x == &y // eql in Lisp
10 == 20 // = in Lisp

Python

>>> x = 10
>>> id(x)
1764365084

>>> y = 10
>>> id(y)
1764365084

>>> x is y // eql in LISP
True

>>> x == y // equal in LISP
True

>>> a = [1,2,3]
>>> a == [1,2,3]
True

>>> a is [1,2,3]
False

>>> id(a)
4359792
>>> id([1,2,3])
4225424

Reference

• Python “is” operator behaves unexpectedly with integers
• What is the semantics of 'is' operator in Python?

Merge two lists in Python

When you merge two lists with Python, you can use the overloaded '+' operator.

a = [1,2,3]
b = [2,3,4]

>>> a + b
[1, 2, 3, 2, 3, 4]

However, when you don't want the duplicate values, you need to use set. '+' operator is not supported in a set, so use union() method.

>>> set(a).union(b)
set([1, 2, 3, 4])

You can convert a set to list with list() method.

>>> list(set(a).union(b))
[1, 2, 3, 4]

Refer to set in python document.

endianness

When programming on Intel CPU, I wondered why the debugger showed the integer values reverse order. The AX register has the value 0x04030201, but when I store the value in memory at 0x1000, the value I see at 0x1000 is 01020304.

That's because Intel uses little endian for their CPU architecture. Little endian stores LSB byte (little byte) first whereas big endian stores MSB first. So the stored value will be 04030201. It's also because we write numbers from left to right. If we have the culture to write from right to left, the little endianness looks OK as we would write 010203040x.

The Internet Protocol defines big-endian as the standard network byte order. When streaming multiple byte data, the first arrived byte is the MSB.

Reference

• wikipedia
• endian-ness of bits and bytes

Deleting elements in map: C++ and Python

C++

std::listlist<int> keys;
// 1. get all the keys that has the elements from bList
for (std::listlist<int>::iterator it = bList.begin(); it != bList.end(); ++it)
{
    for (std::map<string, int>::iterator j = aMap.begin(); j != aMap.end(); ++j)
    {
        if (j->second == *it) keys.push_back(j->first);
    }
}

// 2. remove all the items of the key
for (auto it = keys.begin(); it != keys.end(); ++it)
{
    aMap.erase(*it);
}

std::list<int> keys;
for (auto it = bList.begin(); it != bList.end(); ++it)
{
    for (auto j = aMap.begin(); j != aMap.end(); ++j)
    {
        if (j->second == *it) keys.push_back(j->first);
    }
}

// 3. remove all the items of the key
for (auto it = keys.begin(); it != keys.end(); ++it)
{
    aMap.erase(*it);
}

Python

http://stackoverflow.com/questions/11277432/how-to-remove-a-key-from-dictionary

for e in elements:
    if e in aDict:
        del aDict[e]

for e in elements:
	aDict.pop(e, None)

Alloy example1 - Linked list

Using Alloy, we can make an abstraction model of linked list. Linked list should have the following properties.

• It should have one header as a starting Node, it can link to another Node or null.
• Node has a link which can be another Node or null.
• There should be no cycle.

sig List and Node

The first an second property can be described with “sig”.

one sig List { header: lone Node }

The form of Alloy code looks like a class in OOP, but it’s actually a set. Alloy visually models the member as a label in the arrow, and List as a box.

List has a member named header which can be one of less than one (lone) Node member.

You can think header as a variable name, and Node as the type of the variable.

sig Node { link: lone Node }

Predicate

Then you can have a predicate that is true or false. You can use predicate for describing the acyclic property.

predicate looks like a method or function as it has parameters.

You can describe many ways to describe acyclic, one of them can be “all nodes should not be a link as null”.

And don’t forget about the header, so the link should start from header : link.header.*link

pred acyclic (l : List)

{

        all n : l.header.*link | n !in n.^link

}

l.header.*link means l.header, l.header.link …

n.^link means n.link …

We check the predicate with “run acyclic” as if predicate is an entity to execute.

Alloy returns a list and two nodes that doesn’t have a cycle from the header, but it’s not what we expect.

We missed the fact that all the links from the header should be a Node (that is, has a link of Node or null)

fact

{

        List.header.*link = Node

}

Now, we finally have what we expect.

Equivalence

You can describe the acyclic property in many forms, and you can prove the equivalence between them using Alloy.

pred acyclic2 (l : List)

{

        no l.header or

        some n: l.header.*link | no n.link // n.link is null

}

This predicate says that there can be a link without any element (base case), and some link from List.header.*link should have null. (induction case). Lone means zero or one, no means zero.

pred acyclic3 (l: List)

{

        no (l.header).~link and

        (all n: l.header.*link | lone n.~link)

}

~link means that something that refers link. It means that header’s link is not referenced, and all the links are referenced at most once.

You can run predicates, or you can assert the equivalence.

assert Equiv

{

        all l : List | acyclic[l] <=> acyclic2[l]

}

check Equiv for 5

This command says check the assert with at most 5 elements.

Alloy transforms the code into SAT format, and runs the SAT solver to find a counter example, but it can’t find solution to this SAT problem, so it says they are equivalent.

The idea behind alloy

The importance of abstraction

Software is built on abstractions.

No amount of refactoring can rescue a system built on flawed concepts.

First, you design the abstractions, then you develop its embodiments in code. Unfortunately, this approach rarely works.

The problem is wishful thinking. When you implement the abstractions, they turn out to be incoherent and inconsistent.

There are two solutions to the solution of wishful thinking: “extreme programming” and “formal specification”.

Formal specification has two obstacles: succinct mathematical notations hard to understand and the lack of tools. Theorem provers demand effort more than is feasible for most software projects.

Alloy replaces conventional analysis based on theorem proving with a fully automatic analysis that gives immediate feedback. Unlike theorem proving, this analysis is not “complete”: it examines only a finite space of cases.

Unlike testing, this analysis requires no test cases. The users instead provides a property to be checked.

Except from Daniel Jackson’s Software Abstraction book Chapter 1.

You can see an example from this blog post.

Headless RCP - issue using other plugins

Issues and Solutions

Most of the issues you encounter when you work with headless RCP is highly likely with the “additial plug-ins”.

Missing constraints

Theoretically, if you have run your RCP in IDE, it should run as standalone without a problem. If you have some issues such as “Missing constraints” You can use this site.

http://stackoverflow.com/questions/14366421/the-bundle-xyz-could-not-resolved-reason-missing-constraint-import-package-a/14367997#14367997

Unbound class path container error

You get this error because your java library is wrongly setup.

Go to Add Library session in Build Path/Configure Build Path and set from “Execution environment” to “Workspace …”.

http://stackoverflow.com/questions/6798281/unbound-class-path-container-error-in-eclipse

IllegalStateException: “Workbench has not been created yet”

This error message is misleading, and it actually means “there is some race condition happening”.

You can solve this issue by

1. add -clean option to the parameter
2. Following the “NoClassFoundError case 1”, that is remove all the dependencies and add them again. When you just keep adding dependencies for whatever reasons you may end up this state.

http://waheedtechblog.blogspot.com/2011/11/javalangillegalstateexception-workbench.html

NoClassFoundError case 1

You add this, and that, and suddenly you got an error “NoClassFound”, what you can do is go to the product file, open the Dependencies. Delete all the plug-ins using “Remove” and “Add Required plug-ins” with the application that you are going to make standalone

NoClassFoundError case 2

You successfully created the plugin (utilities for example), but you may have this kind of error “NoClassDefFoundError”, when you execute the plugin that contains other plugin.

First thing you need to understand is that reference is built up already for you.

Lets’s say in the course of plugin development, you add “utilities” in “Dependencies”.

It makes your plugin know how to refer to the utilities during its build, as it finds the utilities, it can build the plugin successfully.

The issue is actually not in the user’s side, but in the provider’s side. You had to export the package in the “utilities” plugin.

SOLID principle in OOP programming

http://en.wikipedia.org/wiki/SOLID_(object-oriented_design)

• Single responsibility principle (SRP)
  • An object should have only a single responsibility.
• Open/closed principle (OCP)
  • Software entitles … should be open for extension, but closed for modification.
• Liskov substitution principle (LSP)
  • Objects in a program should be replaceable with instance of their subtypes without altering the correctness of that program.
• Interface segregation principle (ISP)
  • Many client specific interfaces are better one general purpose interface
• Dependency inversion principle (DIP
  • Do not depend upon concretions. One should depend upon abstraction.

Closure in Java using Anonymous Inner Class - example

Java doesn't support closure, but using Anonymous Inner Class, we can experience closure in Java.

Here is the code.

class Action {
	public void doAction() {
		System.out.println("Hello, world");
	}
}

class A {
	public void doStuff(Action a)
	{
		a.doAction();
	}
    public void method() {
        final int i = 0;

        doStuff(new Action() {
            public void doAction() {
                System.out.println(i);   // or whatever
            }
        });
    }
}

class Closure {
	public static void main(String[] args) {
		new A().method();
	}
}

You should be careful that the local variable in the method() method should be final in order to prevent this error message.

Closure.java:17: error: local variable i is accessed from within inner class; needs to be declared final System.out.println(i); // or whatever ^ 1 error

Instead of giving closure function to doStuff() method, anonymous inner class is given. In its implementation, doStuff() uses methods in A class as if it's closure functions. You can give different code block to doStuff() if necessary.

Compared to python example or lisp example, Java needs full support closure support to make the code simpler and easier to read.

References

• How does java implement inner class closures?

Closure in Lisp - example

This is a code that I borrowed from the book Practical Common Lips for closure in Lisp.

(defun artist-selector (artist)
  #'(lambda (cd) (equal (gets cd :artist) artist)))

In the code, artist-selector returns a closure that closes a variable "artist" over it. We can use this closure as a parameter to "select" function. Lisp supports high order function decades ago to enable it possible. "*db*" is a global variable.

(defun select (selector-fn)
  (remove-if-not selector-fn *db*))

The usage of "select" method is as follows. You can create another closure with different parameter to the "artist-selector" function.

(select (artist-selector "Dixie Chicks"))




References

• 3. Practical: A Simple Database
• Clousre in Python

Closure in Python - example

Closure is a "function (code block)" with a "referencing environment". I interpret it as a function pointer with free variables. In Python terms, a variable defined in an enclosing function is a free variable inside any nested functions.
In this example, printer is a closure as it points to a code block with a free variable of "msg". The variable msg is closed over the printer() function.

def make_printer(msg):
    def printer():
        print msg
    return printer

printer = make_printer('Foo!')
printer()

This printer method is not a closure but nested function, as in this case "msg" variable is nothing but a local variable that is not "closed".

def make_printer(msg):
    def printer(msg=msg):
        print msg
    return printer

printer = make_printer("Foo!")
printer() # "Foo!"
printer("Hello") # --> "Hello"

References

• Closure (computer science)
• Closures in Python

Polymorphism in OOP

Polymorphism in OOP is the ability of objects belonging to different types to respond to method, filed, or property calls of the same name. The caller doesn't have to know the type of the callee so that the exact behavior is determined at run-time.

For strong type languages, polymorphism usually means that type A derives from type B or C, In weakly typed languages types are implicitly polymorphic: duck typing allows polymorphism without inheritance.

There are some disputes whether overloading and overriding is a part of polymorphism.

• Wikipedia explains they are not the same. (Polymorphism is not the same as method overloading or method overriding.1 Polymorphism is only concerned with the application of specific implementations to an interface or a more generic base class.)
• Some of OOP books explain "method overloading" as an example of polymorphism.
• This paper explains overloading as a part of polymorphism
• This post explains that overriding is a part of polymorphism, whereas overloading is not.

Generic programming is also known as "parametric polymorphism".

References

1. Polymorphism in object-oriented programming
2. OBJECT.__class__ is CLASS and "isinstance(OBJECT, CLASS)" for polymorphism in python
3. Duck typing
4. Function overloading
5. Method Overriding
6. Is Method Overloading considered polymorphism?
7. Polymorphism vs Overriding vs Overloading

OBJECT.__class__ is CLASS and "isinstance(OBJECT, CLASS)" for polymorphism in python

The "is" operator in Python compares the two values before and after to check if they point to the same object. Whereass the "is instance" checks if the first parameter (object) is an instance of the second parameter(class).
As a result "dog.__class__ is Animal" returns False, "isinstance(dog, Animal)" returns True.

class Animal:
    pass
    
class Dog (Animal):
    pass
    
dog = Dog()

print dog.__class__ is Animal
print isinstance(dog, Animal)

False True

Things to consider

For C# programmer, this is confusing as "is" operator in C# works as "ins instance" of Python.
In python idiom, you are not encouraged to use type-checking, instead use duck typing instead. In short, you just execute if an object can quack or not instead of checking an object is an instance of a duct that quacks.

class Duck(object): 
    def quack(self):
        print "QUACK!!!"

class DonaldDuck(object): 
    def quack(self):
        print "quack!"


def canQuack(duckType):
    duckType.quack()


duck = Duck()
donaldDuck = DonaldDuck()
canQuack(duck)
canQuack(donaldDuck)

References

How does polymorphism work in Python?

XML file generation using Python's ElementTree

Introduction

When we want to generate XML file from whatever python data structure, ElementTree might be the easiest solution. For example, when we want to transform a list of list . pairList = [ ["A.txt", "B.txt"] … ] into XML format. <root>
<node fileA="A.txt" fileB="B.txt" />
<node ... />
</root>
</code>


Code

We can implement the code as follows.

import xml.etree.ElementTree as ET

root = ET.Element('root')
for pairs in self.pairList:
    #print pairs
    testa = pairs[0]
    testb = pairs[1]
            
    child = ET.Element('node')
    child.attrib['fileA'] = testa
    child.attrib['fileB'] = testb
    root.append(child)
            
file = open(filePath, 'w')

#Create an ElementTree object from the root element
ET.ElementTree(root).write(file)

Explanations

The API names of ElementTree is pretty straightforward. For getting (or creating) an element, you need to use xml.etree.ElementTree.Element (ET.Element when you import with the name ET) API the tag name as a parameter. When you make the element, you can just append (it's also the API name python uses) the newly generated element as a children of the parent element.

You just keep creating the hierarchy using ET.Element() static and append() API.

The element has "attrib" dictionary to store the attributes of the node.

For writing the result into an XML file, you select the root, and use the write method to print out the XML.

Reference

1. Python ElementTree XML Tutorial
2. Processing XML in Python with ElementTree

Adding a new blogspot from MarsEdit.

When you want to setup a new blog connection between blogspot and MarsEdit, this is how you can do it.

The basic idea is to make a setting from MarsEdit as follows:

Procedure

You start from “New Blog”.

Then, you are requested to add google account that relates to the bloodspot.

And that’s it. It automatically gets all the blogs from the blogger so that you can update or add some more.