[wxPython-users] unicode handling

[Dmitriy Baranov]

Thomas Thomas пишет:
> Hi all,
>  
> I have a file with special characters such as "£" etc.  I need to read 
> the file into a list as unicode strings..
> How can I do this.. I tried codecs
>  
> import codecs
> filename='d:/poll/test.XST'

use os.path.join()

> metaHash={}
> infile = codecs.open(filename, "r", encoding='utf-16')
> text = infile.read().split('\n')
> print text
>  
> I am getting the error
>  
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "c:/DOCUME~1/ADMINI~1/LOCALS~1/Temp/python-1928Lij.py", line 9, in ?
>     text = infile.read().split('\n')
>   File "C:\Python23\lib\codecs.py", line 380, in read
>     return self.reader.read(size)
>   File "C:\Python23\lib\encodings\utf_16.py", line 48, in read
>     raise UnicodeError,"UTF-16 stream does not start with BOM"
> UnicodeError: UTF-16 stream does not start with BOM

You are sure what file saved in utf-16 encoding?

http://en.wikipedia.org/wiki/Byte_Order_Mark

for utf-16 BOM - 2 bytes (FE FF)
for utf-8 - 3 bytes (EF BB BF)