CodePongo: python

Debug python Script with pdb

zuohaitao — Tue, 22 Jan 2019 17:30:06 +0200

Usage Launch to debug a script, you can run python -m script_to_debug.py arguments The program will break in the first line of the script. pdb will automatically enter post-mortem debug and restart Insert Statment to Break the Running Script import pdb;pdb.set_trace() if you want to suspend on exception in pdb, try post_mortem method import pdb;pdb.post_mortem() Configure .pdbrc locates home folder or current folder aliases is in this file to include py script into my.pdbrc, use execfile() to run a file containing your Python code. .pdbrc alias p_ for k in sorted(%1.keys()): print "%s%-15s= %-80.80s" % ("%2",k,repr(%1[k])) alias pi p_ %1.__dict__ %1. alias ps pi self alias pl p_ locals() local: alias nl n;;l alias sl s;;l execfile(".pdbrc.py") pdbrc.py in the same directory as the .pdbrc print ".pdbrc.py" Command Arguments to commands must be separated by whitespace multiple commands may be entered on a single, sparated by ;; Help h(elp) [command] Stack Frames w(here) d(own) u(p) Breakpoints b(reak) [filename:]:lineno | function[, condition]] without argument, list all breaks tbreak [filename:]:lineno | function[, condition]] cl(ear) [filename:]:lineno | function[, condition]] disable [bpnumber [bpnumber ...]] enable [bpnumber [bpnumber ...]] ignore bpnumber [count] condition bpnumber [condition] commands [bpnumber] Evaluation s(tep) n(ext) unt(il) r(eturn) c(ont(inue)) j(ump) lineno View l(ist) [first[, last]] a(rgs) p expression Macro alias [name [command]] unalias name PDB Source Code Three main classes in this module, the pdb uses the modules bdb and cmd. cdb.py cmd.py pdb.py Bibliography https://nedbatchelder.com/blog/200704/my_pdbrc.html https://wiki.python.org/moin/PdbRcIdea https://stackoverflow.com/questions/44155444/how-to-write-a-working-pdbrc-file https://docs.python.org/2/library/pdb.html https://stackoverflow.com/questions/45446944/suspend-on-exception-in-pdb

python HTTP Proxy实现和urllib2的使用

zuohaitao — Fri, 21 Nov 2014 17:30:06 +0200

HTTP Proxy HTTP代理 http代理的原理：客户端用代理的ip和port替换服务器的ip和地址建立socket连接；将HTTP请求发送至代理。代理收到http请求，根据http请求的地址与服务器建立连接，将客户端请求转至服务器，将服务器应答转至客户端，客户端除了负责用代理的ip和port，代替服务器的ip和port建立socket，还必须补全Request line（请求行）中的 Path-to-resoure（资源路）为全路径，根据Request header中的Host字段补全。删除请求头中的Proxy-Connection字段代理负责客户端和服务器之间的请求和应答转发。 HTTP代理客户端python实现 HTTP代理服务器python实现 urllib2源码 OpenerDirector 为处理http协议的类，包含三类对象第一类是负责处理请求数据的对象存储在process_request 第二类是负责处理应答数据的对象储存在process_response 第三类是通讯相关对象类存储在handle_open buildopener 函数创建OpenerDirector对象并通过addhandler()将筛选的handler加入对象至OpenDirector default_classes = [ProxyHandler, UnknownHandler, HTTPHandler, HTTPDefaultErrorHandler, HTTPRedirectHandler, FTPHandler, FileHandler, HTTPErrorProcessor] # 所有基础模块 if hasattr(httplib, 'HTTPS'): default_classes.append(HTTPSHandler) skip = set() for klass in default_classes: for check in handlers: if isclass(check): if issubclass(check, klass): skip.add(klass) #有用户自定义的基础模块的继承类则去除基础模块 elif isinstance(check, klass): skip.add(klass) for klass in skip: default_classes.remove(klass) for klass in default_classes: opener.add_handler(klass()) for h in handlers: if isclass(h): h = h() opener.add_handler(h) OpenerDirector的三类对象通过其函数名区分。 protocol_request 如 http_request属于process_request; protocol_response属于process_response; protocol_open属于handle_open def open(self, fullurl, data=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT): # pre-process request meth_name = protocol+"_request" for processor in self.process_request.get(protocol, []): meth = getattr(processor, meth_name) req = meth(req) response = self._open(req, data) # post-process response meth_name = protocol+"_response" for processor in self.process_response.get(protocol, []): meth = getattr(processor, meth_name) response = meth(req, response) return response def _open(self, req, data=None): protocol = req.get_type() result = self._call_chain(self.handle_open, protocol, protocol + '_open', req) def _call_chain(self, chain, kind, meth_name, *args): # Handlers raise an exception if no one else should try to handle # the request, or return None if they can't but another handler # could. Otherwise, they return the response. handlers = chain.get(kind, ()) for handler in handlers: func = getattr(handler, meth_name) result = func(*args) if result is not None: return result urllib2 代理相关 Request self.host为设置为代理地址 self. __ original 为真正地址 ProxyHandler 初始化时，会增加protocolopen方法，该方法调用proxyopen进行设置代理 for type, url in proxies.items(): setattr(self, '%s_open' % type, lambda r, proxy=url, type=type, meth=self.proxy_open: \ meth(r, proxy, type)) 注意设置代理时，代理模块会读取系统的代理设置 def proxy_open(self, req, proxy, type): if req.host and proxy_bypass(req.host): #注意：proxy_bypass return None proxy_bypass 位于urllib模块中,会读取保存在操作系统中的代理设置，windows在注册表中 internetSettings = _winreg.OpenKey(_winreg.HKEY_CURRENT_USER, r'Software\Microsoft\Windows\CurrentVersion\Internet Settings') proxyEnable = _winreg.QueryValueEx(internetSettings, 'ProxyEnable')[0] if proxyEnable: # Returned as Unicode but problems if not converted to ASCII proxyServer = str(_winreg.QueryValueEx(internetSettings, 'ProxyServer')[0])

The Cahche module in Planet

zuohaitao — Fri, 21 Feb 2014 17:30:06 +0200

modules +--------------+ +-----+ |planet.Channel|-+-is--|cache| +----------+---+ \ +-----+ \ \ +------+ \ +-have-+-|dbhash| \ / +------+ +--------+ dbhash Planet(http://www.planetplanet.org/) uses the hashdb module to cache the feeded data. The dbhash module provides a function to open databases using the BSD db library. This module mirrors the interface of the other Python database modules that provide access to DBM-style databases. The bsddb module is required to use dbhash. """output the hashdb""" import sys import dbhash db = dbhash.open(sys.argv[1], 'r') print db.first() for i in xrange(1, len(db)): print db.next() db.close() The dbm library was a simple database engine, originally written by Ken Thompson and released by AT&T in 1979. The name is a three letter acronym for database manager, and can also refer to the family of database engines with APIs and features derived from the original dbm. planet.Channel and cache modules the module description is very clear and easy to know, so you can use help() to show the module description and read it. database file the database is key/value database * " keys" is the key of the root item. the root item's value is the keys of the channel properties that is splited by blank space. the key in that there is not blank space is the key of the news item. the news item's value is the keys of the news properties that is splited by blank space. the news property's key is the unit of the news item's key and the part of the news item's value. the channel property item and the news property item the key of that includes the string " type" is provided the type of the news property's value. For example: the RSS: CodePongo: json Thu, 22 Aug 2013 18:16:02 +0200 cJSON source analysis the database: (' keys', 'lastbuilddate title') ('lastbuilddate type', 'string') ('lastbuilddate', 'Thu, 22 Aug 2013 18:16:02 +0200') ('title type', 'string') ('title', 'CodePongo: json') ('http://codepongo.com/blog/3uhyc1/ title type', 'string') ('http://codepongo.com/blog/3uhyc1/ title', 'cJSON source analysis') ('http://codepongo.com/blog/3uhyc1/', 'title') reference planet home page planet introduction dbhash module database

Project-Generator

zuohaitao — Fri, 29 Nov 2013 17:30:06 +0200

introduction https://github.com/Piot/Project-Generator Project-Generator makes the native IDE project files. it supports Visual Studio(windows), Xcode(mac os x), makefile(linux) and so on. it likes the GYP. but, it is smaller, simpler and easier to understand than GYP. its config option in a XML file is more readable than GYP input format file. process in generate.py parses arguments project module creates target project project parser module initials target project with xml node generator module write to the project files with project write module project.py class SourceFileNode - all source files class Dependecy - dependent projects class Define - macros class Settings - project settings likes include paths, defines, dependecies, libraries,framewoorks and so on class configuration - inherit from Settings, it is a build Settings with a name such as "debug", "relase" and so on. class Project - the project object relationship Project+- depandencies +- configurations +- settings-+-paths +-defines +-denpendecies +-libraries +-frameworks project_object.py project_parser.py initial project object with xml node project_path.py operation about paths project_write.py class ProjectFileCreator:a Factory in where ProjectFileOutput is made. class ProjectOutput:a controller of project file's indent class ProjectFileOutput:inherit from ProjectOutput wrapper of file operation generator codeblocks.py codelite.py makefile.py makefile in Linux platform visualc.py Visual Studio in Windows platform xcode.py Xcode in Mac OS X platform

zjdict 命令行字典

zuohaitao — Mon, 22 Oct 2012 17:30:06 +0200

a command line word dictionary Function 功能 green application 绿色软件无需安装解压即用 new word file as text format 文本格式生词本 command user interface 命令行交互 local dictionary. 离线词典 you can translate word by local startdict dictionary 本地使用星际译王词典 translate on line. 在线翻译 support baidu online translate and youdao online translate. 支持百度词典，有道词典在线翻译 mode 多模式 translate mode 翻译模式 search mode 查找模式 matching 匹配方式 regular expression 支持正则表达式 snapshot 截图 command 命令 ,save save to 'new word' file保存至生词本 ,search 查找 ,search search mode 查找模式 ,quit or ,exit exit 退出 develop 开发 source code 源代码 @github the newest @googlecode mirror third party library 第三方库 pystardict @sourceforge @github getch