/usr/lib/python2.6/site-packages/ndg/httpsclient
Edit: /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyc (13216B)
Ñò
˜KVc @ s¨ d Z d Z d Z d Z d Z d Z d Z d d k Z d d k l Z d d k
Z
d d k Z e i d
d j o¡ d d k
i Z d d k i Z d d k l Z d d
k l Z d d k l Z d d k l Z d d k l Z d d k l Z d d k l! Z" d d k# i$ Z% n• d d k& Z d d k' Z d d k( l Z d d
k( l Z d d k( l Z d d k( l Z d d k( l Z d d k( l Z d d k( l! Z" d d k) Z% d d k* l+ Z+ d d k, l- Z- d d k. l/ Z/ e i0 e1 ƒ Z2 d e f d „ ƒ YZ3 d e4 f d „ ƒ YZ5 d d d „ Z7 d d d „ Z8 d d d „ Z9 d d d „ Z: d d „ Z; d „ Z< d e= f d! „ ƒ YZ> d" „ Z? e1 d# j o e i@ ƒ e? ƒ n d S($ s` Utilities using NDG HTTPS Client, including a main module that can be used to
fetch from a URL.
s
R B Wilkinsons 09/12/11s2 (C) 2011 Science and Technology Facilities Councils- BSD - see LICENSE file in top-level directorys Philip.Kershaw@stfc.ac.uks $Id$iÿÿÿÿN( t OptionParseri i ( t Request( t HTTPHandler( t HTTPCookieProcessor( t HTTPBasicAuthHandler( t HTTPPasswordMgrWithDefaultRealm( t ProxyHandler( t HTTPError( t build_opener( t HTTPSContextHandler( t ssl_context_utilt AccumulatingHTTPCookieProcessorc B s e Z d Z d „ Z e Z RS( st Cookie processor that adds new cookies (instead of replacing the existing
ones as HTTPCookieProcessor does)
c C s³ d } t | i ƒ | i h | i | i ƒ } | i i | ƒ | i | ƒ } | o\ | i | ƒ o8 | i | ƒ } d i | | g ƒ } | i
| | ƒ q¯ | i
| | ƒ n | S( s¹ Processes cookies for a HTTP request.
@param request: request to process
@type request: urllib2.Request
@return: request
@rtype: urllib2.Request
t Cookies ; ( t Request_t get_full_urlt datat origin_req_hostt unverifiablet cookiejart add_cookie_headert
get_headert
has_headert joint add_unredirected_header( t selft requestt COOKIE_HEADER_NAMEt tmp_requestt new_cookiest old_cookiest merged_cookies( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyt http_request3 s ( t __name__t
__module__t __doc__R t
https_request( ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyR / s t
URLFetchErrorc B s e Z d Z RS( s Error fetching content from URL( R R! R" ( ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyR$ Q s c C sf t | | d | d | ƒ\ } } } | o+ | t i j o | i ƒ } | i ƒ | St | ƒ ‚ d S( sá Returns data retrieved from a URL.
@param url: URL to attempt to open
@type url: basestring
@param config: SSL context configuration
@type config: Configuration
@return data retrieved from URL or None
R t handlersN( t open_urlt http_client_t OKt readt closeR$ ( t urlt configR R% t return_codet return_messaget responset return_data( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyt fetch_from_urlU s
c
C sŠ t | | d | d | ƒ\ } } } | t i j o@ | i ƒ } | i ƒ t | d ƒ } | i | ƒ | i ƒ n | | | t i j f S( s¨ Writes data retrieved from a URL to a file.
@param url: URL to attempt to open
@type url: basestring
@param config: SSL context configuration
@type config: Configuration
@param output_file: output file
@type output_file: basestring
@return: tuple (
returned HTTP status code or 0 if an error occurred
returned message
boolean indicating whether access was successful)
R R% t w( R& R' R( R) R* t opent write(
R+ R, t output_fileR R% R- R. R/ R0 t outfile( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyt fetch_from_url_to_filef s
c C sP t | | d | d | ƒ\ } } } | o | t i j o | St | ƒ ‚ d S( sœ Returns data retrieved from a URL.
@param url: URL to attempt to open
@type url: basestring
@param config: SSL context configuration
@type config: Configuration
@param data: HTTP POST data
@type data: str
@param handlers: list of custom urllib2 handlers to add to the request
@type handlers: iterable
@return: data retrieved from URL or None
@rtype: file derived type
R R% N( R& R' R( R$ ( R+ R, R R% R- R. R/ ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyt fetch_stream_from_url s
c
C s | i o d n d } | i o
| i } n
t i ƒ } t | ƒ } | p
g } n | i | ƒ | i o; t d | ƒ } t | i d | ƒ} | i | | g ƒ n | i
oP t t ƒ ƒ } | i
d d d | d | i d d | i d ƒ | i | ƒ n t | | i ƒ p$ | i t h ƒ ƒ t i d ƒ n9 | i o. | i t | i ƒ ƒ t i d | i ƒ n t d
| i | Π}
| i } | d j o
h } n t | | | ƒ } d }
d } d } |
i | ƒ } yl |
i | ƒ } | i } | i }
t i t i ƒ o4 x1 t | ƒ D] \ } } t i d | | ƒ q Wn Wnµ t j
oO } | i }
d
| i } t i t i ƒ o t i d | i | i ƒ qþn\ t j
oO } d
| i! ƒ } t i t i ƒ o# d d k" } t i | i# ƒ ƒ qþn X|
| | f S( sÞ Attempts to open a connection to a specified URL.
@param url: URL to attempt to open
@param config: SSL context configuration
@type config: Configuration
@param data: HTTP POST data
@type data: str
@param handlers: list of custom urllib2 handlers to add to the request
@type handlers: iterable
@return: tuple (
returned HTTP status code or 0 if an error occurred
returned message or error description
response object)
i i t
debuglevelt realmt urit usert passwds Not using proxys Configuring proxies: %st ssl_contextt s %s : %ss Error: %ss %s %siÿÿÿÿN($ t debugt cookiet
cookiejar_t CookieJarR t appendt HTTPHandler_R R> t extendt http_basicautht HTTPBasicAuthHandler_t HTTPPasswordMgrWithDefaultRealm_t add_passwordt Nonet httpautht _should_use_proxyt no_proxyt
ProxyHandler_t logt proxiesR t headersR
R3 t msgt codet isEnabledFort loggingt DEBUGt enumeratet
HTTPError_t Exceptiont __str__t tracebackt
format_exc( R+ R, R R% R9 t cjt cookie_handlert http_handlert
https_handlert auth_handlert openerRR R R- R. R/ t indexRA t excR\ ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyR& ” sj
! c C s• | d j o t i i d d ƒ } n | } t i t | ƒ ƒ } xM g } | i d ƒ D] } | | i ƒ qX ~ D] } | i | j o t
Sqr Wt S( sØ Determines whether a proxy should be used to open a connection to the
specified URL, based on the value of the no_proxy environment variable.
@param url: URL
@type url: basestring or urllib2.Request
RN R? t ,N( RK t ost environt gett urlparse_t urlparset _url_as_stringt splitt stript hostnamet Falset True( R+ RN t no_proxy_effectivet urlObjt _[1]t ht np( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyRM ö s
1 c C sJ t | t ƒ o | i ƒ St | t ƒ o | St d t t f ƒ ‚ d S( sÍ Returns the URL string from a URL value that is either a string or
urllib2.Request..
@param url: URL
@type url: basestring or urllib2.Request
@return: URL string
@rtype: basestring
s Expected type %r or %rN( t
isinstanceR
R t strt TypeError( R+ ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyRl s t
Configurationc B s) e Z d Z e d d d d d d „ Z RS( s Connection configuration.
c C sC | | _ | | _ | | _ | | _ | | _ | | _ | | _ d S( sÔ
@param ssl_context: SSL context to use with this configuration
@type ssl_context: OpenSSL.SSL.Context
@param debug: if True, output debugging information
@type debug: bool
@param proxies: proxies to use for
@type proxies: dict with basestring keys and values
@param no_proxy: hosts for which a proxy should not be used
@type no_proxy: basestring
@param cookie: cookies to set for request
@type cookie: cookielib.CookieJar (python 3 - http.cookiejar)
@param http_basicauth: http authentication, or None
@type http_basicauth: tuple of (username,password)
@param headers: http headers
@type headers: dict
N( R> R@ RQ RN RA RG RR ( R R> R@ RQ RN RA RG RR ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyt __init__ s N( R R! R" Rp RK R{ ( ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyRz s c C sè t d d ƒ } | i d d d d d d d t i i d
ƒ d d ƒ| i d
d d d d d d d8 d d ƒ| i d d d d d d d d8 d d ƒ| i d d d d d d d t d d ƒ| i d d d d d d d d8 d d ƒ| i d d! d d" d d d d8 d d# ƒ| i d$ d% d d d d& d t d d' ƒ| i d( d) d d* d d+ d d8 d d, ƒ| i d- d d. d d/ d d0 d d1 ƒ| i ƒ \ } } t | ƒ d2 j o | i d3 ƒ n | d4 } | i
o t i ƒ i
t i ƒ n | i o# t i i | i ƒ o
| i } n d8 } | i o# t i i | i ƒ o
| i } n d8 } | i o# t i i | i ƒ o
| i } n d8 } | i } | i o? t i i | i ƒ o) t | i ƒ } | i ƒ } | i ƒ n d8 } | i o | i i d5 d2 ƒ }
n d8 }
h } | i oF xC | i D]4 } | i d5 d2 ƒ \ }
} | i ƒ | |
i ƒ
R, R- R. ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyt main7 sœ
t __main__(A R" t
__author__t __date__t
__copyright__t __license__t __contact__t __revision__RV t optparseR Rg t syst version_infot http.cookiejarR RB t http.clientt clientR' t urllib.requestR R
R RE R t HTTPCookieProcessor_R RH R RI R RO t urllib.errorR RY t urllib.parset parseRj t cookielibt httplibt urllib2Rk t$ ndg.httpsclient.urllib2_build_openerR t ndg.httpsclient.httpsR t ndg.httpsclientR
R’ R RP R RZ R$ RK R1 R7 R8 R& RM Rl t objectRz Rž t basicConfig( ( ( s9 /usr/lib/python2.6/site-packages/ndg/httpsclient/utils.pyt s` "b e