Error when fetching project dependencies as wheels and generate ABOUT files

[chinyeungli]

When I perform this step: python etc/scripts/bootstrap.py -r requirements.txt -r requirements-dev.txt --with-deps

in CMD

==> FETCHING ABOUT AND LICENSE FILES
Traceback (most recent call last):
  File "etc/scripts/bootstrap.py", line 212, in <module>
    bootstrap()
  File "C:\Users\thoma\git\spats\venv\lib\site-packages\click\core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\thoma\git\spats\venv\lib\site-packages\click\core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "C:\Users\thoma\git\spats\venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\thoma\git\spats\venv\lib\site-packages\click\core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "etc/scripts/bootstrap.py", line 199, in bootstrap
    utils_thirdparty.add_fetch_or_update_about_and_license_files(dest_dir=thirdparty_dir)
  File "C:\Users\thoma\git\spats\etc\scripts\utils_thirdparty.py", line 2489, in add_fetch_or_update_about_and_license_files
    remote_dist.load_remote_about_data()
  File "C:\Users\thoma\git\spats\etc\scripts\utils_thirdparty.py", line 761, in load_remote_about_data
    about_text = fetch_content_from_path_or_url_through_cache(self.about_download_url)
  File "C:\Users\thoma\git\spats\etc\scripts\utils_thirdparty.py", line 2011, in fetch_content_from_path_or_url_through_cache
    return cache.get(path_or_url=path_or_url, as_text=as_text)
  File "C:\Users\thoma\git\spats\etc\scripts\utils_thirdparty.py", line 1846, in get
    fo.write(content)
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python36\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u0142' in position 201: character maps to <undefined>

in WSL2

==> FETCHING SOURCE DISTRIBUTIONS
Finding sources for: extractcode-7z==16.5.210531: No sources found
Finding sources for: extractcode-libarchive==3.5.1.210531: No sources found
Finding sources for: typecode-libmagic==5.39.210531: No sources found
==> FETCHING ABOUT AND LICENSE FILES
Traceback (most recent call last):
  File "etc/scripts/bootstrap.py", line 212, in <module>
    bootstrap()
  File "/mnt/c/Users/thoma/git/spats/venv/lib/python3.6/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/mnt/c/Users/thoma/git/spats/venv/lib/python3.6/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/mnt/c/Users/thoma/git/spats/venv/lib/python3.6/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/mnt/c/Users/thoma/git/spats/venv/lib/python3.6/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "etc/scripts/bootstrap.py", line 199, in bootstrap
    utils_thirdparty.add_fetch_or_update_about_and_license_files(dest_dir=thirdparty_dir)
  File "/mnt/c/Users/thoma/git/spats/etc/scripts/utils_thirdparty.py", line 2422, in add_fetch_or_update_about_and_license_files
    local_dist.load_about_data(dest_dir=dest_dir)
  File "/mnt/c/Users/thoma/git/spats/etc/scripts/utils_thirdparty.py", line 752, in load_about_data
    about_data['notice_text'] = fi.read()
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 1180: invalid start byte

[chinyeungli]

I've made some changes about the encoding when doring read and write in the utils_thirdparty.py
and it's working on the WSL2 side. I will need to check with the windows' CMD side before I push the update.

[pombredanne]

@chinyeungli I just saw this! thanks! Are we all set?