Name	Size	Mode	Actions
big5freq.cpython-37.opt-1.pyc	27139	0644	editdlrm
big5freq.cpython-37.pyc	27139	0644	editdlrm
big5prober.cpython-37.opt-1.pyc	1080	0644	editdlrm
big5prober.cpython-37.pyc	1080	0644	editdlrm
chardistribution.cpython-37.opt-1.pyc	6266	0644	editdlrm
chardistribution.cpython-37.pyc	6266	0644	editdlrm
charsetgroupprober.cpython-37.opt-1.pyc	2187	0644	editdlrm
charsetgroupprober.cpython-37.pyc	2187	0644	editdlrm
charsetprober.cpython-37.opt-1.pyc	3397	0644	editdlrm
charsetprober.cpython-37.pyc	3397	0644	editdlrm
codingstatemachine.cpython-37.opt-1.pyc	2844	0644	editdlrm
codingstatemachine.cpython-37.pyc	2844	0644	editdlrm
compat.cpython-37.opt-1.pyc	315	0644	editdlrm
compat.cpython-37.pyc	315	0644	editdlrm
cp949prober.cpython-37.opt-1.pyc	1087	0644	editdlrm
cp949prober.cpython-37.pyc	1087	0644	editdlrm
enums.cpython-37.opt-1.pyc	2578	0644	editdlrm
enums.cpython-37.pyc	2578	0644	editdlrm
escprober.cpython-37.opt-1.pyc	2565	0644	editdlrm
escprober.cpython-37.pyc	2565	0644	editdlrm
escsm.cpython-37.opt-1.pyc	7026	0644	editdlrm
escsm.cpython-37.pyc	7026	0644	editdlrm
eucjpprober.cpython-37.opt-1.pyc	2373	0644	editdlrm
eucjpprober.cpython-37.pyc	2373	0644	editdlrm
euckrfreq.cpython-37.opt-1.pyc	12023	0644	editdlrm
euckrfreq.cpython-37.pyc	12023	0644	editdlrm
euckrprober.cpython-37.opt-1.pyc	1088	0644	editdlrm
euckrprober.cpython-37.pyc	1088	0644	editdlrm
euctwfreq.cpython-37.opt-1.pyc	27143	0644	editdlrm
euctwfreq.cpython-37.pyc	27143	0644	editdlrm
euctwprober.cpython-37.opt-1.pyc	1088	0644	editdlrm
euctwprober.cpython-37.pyc	1088	0644	editdlrm
gb2312freq.cpython-37.opt-1.pyc	19067	0644	editdlrm
gb2312freq.cpython-37.pyc	19067	0644	editdlrm
gb2312prober.cpython-37.opt-1.pyc	1096	0644	editdlrm
gb2312prober.cpython-37.pyc	1096	0644	editdlrm
hebrewprober.cpython-37.opt-1.pyc	2930	0644	editdlrm
hebrewprober.cpython-37.pyc	2930	0644	editdlrm
jisfreq.cpython-37.opt-1.pyc	22095	0644	editdlrm
jisfreq.cpython-37.pyc	22095	0644	editdlrm
jpcntx.cpython-37.opt-1.pyc	37974	0644	editdlrm
jpcntx.cpython-37.pyc	37974	0644	editdlrm
langbulgarianmodel.cpython-37.opt-1.pyc	23588	0644	editdlrm
langbulgarianmodel.cpython-37.pyc	23588	0644	editdlrm
langcyrillicmodel.cpython-37.opt-1.pyc	29044	0644	editdlrm
langcyrillicmodel.cpython-37.pyc	29044	0644	editdlrm
langgreekmodel.cpython-37.opt-1.pyc	23546	0644	editdlrm
langgreekmodel.cpython-37.pyc	23546	0644	editdlrm
langhebrewmodel.cpython-37.opt-1.pyc	22175	0644	editdlrm
langhebrewmodel.cpython-37.pyc	22175	0644	editdlrm
langhungarianmodel.cpython-37.opt-1.pyc	23577	0644	editdlrm
langhungarianmodel.cpython-37.pyc	23577	0644	editdlrm
langthaimodel.cpython-37.opt-1.pyc	22154	0644	editdlrm
langthaimodel.cpython-37.pyc	22154	0644	editdlrm
langturkishmodel.cpython-37.opt-1.pyc	22177	0644	editdlrm
langturkishmodel.cpython-37.pyc	22177	0644	editdlrm
latin1prober.cpython-37.opt-1.pyc	2887	0644	editdlrm
latin1prober.cpython-37.pyc	2887	0644	editdlrm
mbcharsetprober.cpython-37.opt-1.pyc	2192	0644	editdlrm
mbcharsetprober.cpython-37.pyc	2192	0644	editdlrm
mbcsgroupprober.cpython-37.opt-1.pyc	1083	0644	editdlrm
mbcsgroupprober.cpython-37.pyc	1083	0644	editdlrm
mbcssm.cpython-37.opt-1.pyc	15638	0644	editdlrm
mbcssm.cpython-37.pyc	15638	0644	editdlrm
sbcharsetprober.cpython-37.opt-1.pyc	2945	0644	editdlrm
sbcharsetprober.cpython-37.pyc	2945	0644	editdlrm
sbcsgroupprober.cpython-37.opt-1.pyc	1573	0644	editdlrm
sbcsgroupprober.cpython-37.pyc	1573	0644	editdlrm
sjisprober.cpython-37.opt-1.pyc	2399	0644	editdlrm
sjisprober.cpython-37.pyc	2399	0644	editdlrm
universaldetector.cpython-37.opt-1.pyc	5789	0644	editdlrm
universaldetector.cpython-37.pyc	5789	0644	editdlrm
utf8prober.cpython-37.opt-1.pyc	1930	0644	editdlrm
utf8prober.cpython-37.pyc	1930	0644	editdlrm
version.cpython-37.opt-1.pyc	399	0644	editdlrm
version.cpython-37.pyc	399	0644	editdlrm
__init__.cpython-37.opt-1.pyc	804	0644	editdlrm
__init__.cpython-37.pyc	804	0644	editdlrm

B <_9Y�0�@s�dZd
dl
Z
d
dlZd
dlZddlmZ
ddlmZmZm Z
ddl mZ
ddlm Z
ddlmZ
dd lmZ
Gd d�de�ZdS)a
Module containing the UniversalDetector detector class, which is the primary class a user of ``chardet`` should use. :author: Mark Pilgrim (initial port to Python) :author: Shy Shalom (original C code) :author: Dan Blanchard (major refactoring for 3.0) :author: Ian Cordasco �N�
)
�CharSetGroupProber)� InputState�LanguageFilter�ProbingState)
�EscCharSetProber)
�Latin1Prober)
�MBCSGroupProber)
�SBCSGroupProberc @sneZ
dZd
ZdZe�d�
Ze�d�
Ze�d�
Z dddd d ddd d�Z ejf
dd�
Z dd�Zdd�Zdd�ZdS)�UniversalDetectoraq
The ``UniversalDetector`` class underlies the ``chardet.detect`` function and coordinates all of the different charset probers. To get a ``dict`` containing an encoding and its confidence, you can simply run: .. code:: u = UniversalDetector() u.feed(some_bytes) u.close() detected = u.result g�������?s[�-�]s(|~{)s[�-�]zWindows-1252zWindows-1250zWindows-1251zWindows-1256zWindows-1253zWindows-1255zWindows-1254zWindows-1257)z iso-8859-1z iso-8859-2z iso-8859-5z iso-8859-6z iso-8859-7z iso-8859-8z iso-8859-9ziso-8859-13cCsNd|_g|_
d|_d|_d|_d|_d|_|
|_t� t �
|_d|_|� �
dS)
N)�_esc_charset_prober�_charset_probers�result�done� _got_data�_input_state� _last_char�lang_filter�loggingZ getLogger�__name__�logger�_has_win_bytes�reset)�selfr�r�J/opt/alt/python37/lib/python3.7/site-packages/chardet/universaldetector.py�__init__Qs










zUniversalDetector.__init__c
CsZd
dd
d�|_d|_
d|_d|_tj|_d|_|jr>|j� �
x|j D]}
|
� �
qFWd
S)z� Reset the UniversalDetector and all of its probers back to their initial states. This is called by ``__init__``, so you only need to call this directly in between analyses of different documents. Ng)�encoding� confidence�languageF�)rrrrr� PURE_ASCIIrrrrr )r�proberrrrr^s








zUniversalDetector.resetcCs>|jr d
St
|
�
sd
St|
t�s(t|
�
}
|js�|
�tj�
rJdddd�|_nv|
�tj tj f�
rldddd�|_nT|
�d�
r�dddd�|_n:|
�d �
r�d ddd�|_n |
�tjtjf�
r�dddd�|_d|_|jd d
k r�d|_d
S|j tjk�
r.|j�|
�
�
rtj|_ n*|j tjk�
r.|j�|j|
�
�
r.tj|_ |
dd
�|_|j tjk�
r�|j�
s^t|j�
|_|j�|
�
tjk�r:|jj|j��|jjd�|_d|_n�|j tjk�r:|j�
s�t |j�
g
|_|jt!j"@�
r�|j�#t$��

|j�#t%��

x@|jD]6}|�|
�
tjk�
r�|j|��|jd�|_d|_P�
q�W|j&�|
�
�r:d|_'d
S)a�
Takes a chunk of a document and feeds it through all of the relevant charset probers. After calling ``feed``, you can check the value of the ``done`` attribute to see if you need to continue feeding the ``UniversalDetector`` more data, or if it has made a prediction (in the ``result`` attribute). .. note:: You should always call ``close`` when you're done feeding in your document if ``done`` is not already ``True``. Nz UTF-8-SIGg�?�)rrrzUTF-32s��zX-ISO-10646-UCS-4-3412s��zX-ISO-10646-UCS-4-2143zUTF-16Tr�����)(r�len� isinstance� bytearrayr� startswith�codecs�BOM_UTF8r�BOM_UTF32_LE�BOM_UTF32_BE�BOM_LE�BOM_BErrr!�HIGH_BYTE_DETECTOR�search� HIGH_BYTE�ESC_DETECTORrZ ESC_ASCIIrrr�feedrZFOUND_IT�charset_name�get_confidencerr r rZNON_CJK�appendr r�WIN_BYTE_DETECTORr)rZbyte_strr"rrrr3os|










































zUniversalDetector.feedc
Cs�
|jr|j
Sd
|_|js&|j�d�

n�|jtjkrBdddd�|_
n�|jtjkr�d}
d}d}x,|j D]"}|slqb|� �}
|
|krb|
}|}qbW|r�||jkr�|j}|j� �}|� �}|�d �
r�|jr�|j�||�}|||jd�|_
|j��tjk�
rz|j
d dk�
rz|j�d�

xn|j D]d}|�
s �
qt|t��
rZxF|jD] }|j�d|j|j|� ��
�
q4Wn|j�d|j|j|� ��
�
qW|j
S) z� Stop analyzing the current document and come up with a final prediction. :returns: The ``result`` attribute, a ``dict`` with the keys `encoding`, `confidence`, and `language`. Tzno data received!�asciig�?r#)rrrNgziso-8859rz no probers hit minimum thresholdz%s %s confidence = %s)rrrr�debugrrr!r1r r5�MINIMUM_THRESHOLDr4�lowerr(r�ISO_WIN_MAP�getrZgetEffectiveLevelr�DEBUGr&rZprobers) rZprober_confidenceZmax_prober_confidenceZ max_proberr"r4Zlower_charset_namerZgroup_proberrrr�close�s` 








































zUniversalDetector.closeN)r� __module__�__qualname__�__doc__r:�re�compiler/r2r7r<rZALLrrr3r?rrrrr3s"










 mr)rBr)rrCZcharsetgroupproberrZenumsrrrZ escproberrZlatin1proberrZmbcsgroupproberr Zsbcsgroupproberr �objectrrrrr�$s