Go to file
Kory Maincent d6d7cbb8e0 support/testing: don't fail on tests emitting invalid utf-8 sequences
When booting under EFI, grub2 will output a nice and shiny boot menu,
using extended ASCII characters (in the [0x80..0xFF] range), namely
CP437 [0], on the assumption that the VGA BIOS is a real one and has the
corresponding (and only!) font, as is the case on real hardware.

However, when run in our runtime test infrastructure, this triggers the
infamous python UnicodeDecodeError exception:

    Traceback (most recent call last):
      [...]
        emulator.login()
      File "[...]/buildroot/support/testing/infra/emulator.py", line 89, in login
        index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
      File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
        return self.expect_list(compiled_pattern_list,
      File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
        return exp.expect_loop(timeout)
      File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
        incoming = spawn.read_nonblocking(spawn.maxread, timeout)
      File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
        return super(spawn, self).read_nonblocking(size)
      File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
        s = self._decoder.decode(s, final=False)
      File "/usr/lib/python3.8/codecs.py", line 322, in decode
        (result, consumed) = self._buffer_decode(data, self.errors, final)
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte

Grub2 is not wrong in emitting those chars, and basically we should not
expect the packages we test to always emit correct UTF-8 sequences; at
the very least, this should not cause the test infra to fail.

We fix that by telling pexpect.spawn to "fix" such invalid sequences by
replacing them with the suitable Unicode character, U+FFFD REPLACEMENT
CHARACTER.

[0] https://en.wikipedia.org/wiki/Code_page_437
[1] https://docs.python.org/3/library/codecs.html#error-handlers

Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
  - don't change encoding, use codec_errors
  - rewrite commit log accordingly
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-10-03 14:26:12 +02:00
arch arch: add BR2_ARCH_NEEDS_GCC_AT_LEAST_11 2021-06-20 18:46:51 +02:00
board board, boot, package: remove usage of startup.nsh in EFI partition 2021-09-27 21:27:02 +02:00
boot boot/grub2: add support to build multiple Grub2 configurations in the same build 2021-09-27 21:36:06 +02:00
configs configs/stm32mp157c_odyssey: new defconfig 2021-09-11 18:15:55 +02:00
docs docs/manual/using-buildroot-debugger: suggest '-ix' iso '-x' when loading gdbinit 2021-09-21 22:43:03 +02:00
fs fs/iso9660: add support for hybrid image using Grub2 on BIOS and EFI 2021-09-29 22:58:57 +02:00
linux {linux, linux-headers}: bump 4.{4, 9, 14, 19}.x / 5.{4, 10, 13, 14}.x series 2021-09-12 13:43:16 +02:00
package package/python-xlsxwriter: bump to version 3.0.1 2021-10-03 10:22:51 +02:00
support support/testing: don't fail on tests emitting invalid utf-8 sequences 2021-10-03 14:26:12 +02:00
system system: support br2-external init systems 2020-10-14 22:48:42 +02:00
toolchain Revert "make: support: use command -v' instead of which'" 2021-10-01 20:09:58 +02:00
utils utils/scanpypi: remove python2 compatibility imports 2021-09-22 21:40:43 +02:00
.defconfig arch: remove support for sh64 2016-09-08 22:15:15 +02:00
.flake8 Revert ".flake8: fix check for 80/132 columns" 2021-01-02 17:38:20 +01:00
.gitignore
.gitlab-ci.yml utils/test-pkg: add gitlab-ci support 2021-08-28 17:03:13 +02:00
CHANGES Update for 2021.02.5 2021-09-16 22:33:46 +02:00
Config.in Config.in: disable Fortify Source for microblaze 2021-08-23 23:08:05 +02:00
Config.in.legacy package/strongswan: add options to select EAP plugins 2021-09-28 22:43:51 +02:00
COPYING
DEVELOPERS package/mstpd: new package 2021-09-26 23:37:08 +02:00
Makefile Revert "make: support: use command -v' instead of which'" 2021-10-01 20:09:58 +02:00
Makefile.legacy Remove BR2_DEPRECATED 2016-10-15 23:14:45 +02:00
README docs: move the IRC channel away from Freenode 2021-05-29 22:16:23 +02:00

Buildroot is a simple, efficient and easy-to-use tool to generate embedded
Linux systems through cross-compilation.

The documentation can be found in docs/manual. You can generate a text
document with 'make manual-text' and read output/docs/manual/manual.text.
Online documentation can be found at http://buildroot.org/docs.html

To build and use the buildroot stuff, do the following:

1) run 'make menuconfig'
2) select the target architecture and the packages you wish to compile
3) run 'make'
4) wait while it compiles
5) find the kernel, bootloader, root filesystem, etc. in output/images

You do not need to be root to build or run buildroot.  Have fun!

Buildroot comes with a basic configuration for a number of boards. Run
'make list-defconfigs' to view the list of provided configurations.

Please feed suggestions, bug reports, insults, and bribes back to the
buildroot mailing list: buildroot@buildroot.org
You can also find us on #buildroot on OFTC IRC.

If you would like to contribute patches, please read
https://buildroot.org/manual.html#submitting-patches