• Status New
  • Percent Complete
    0%
  • Task Type Bug Report
  • Category Packages
  • Assigned To No-one
  • Operating System pentium4
  • Severity Low
  • Priority Medium
  • Reported Version
  • Due in Version Undecided
  • Due Date Undecided
  • Votes
  • Private
Attached to Project: Arch Linux 32
Opened by Andreas Baumann - 26.01.2021

FS#136 - Haskell probably completely fails to rebuild

Configuring attoparsec-0.13.2.4...
Error:
    The following packages are broken because other packages they depend on are missing. These broken packages must be rebuilt before they can be used.
installed package tasty-1.3.1 is broken due to missing package ansi-terminal-0.11-2SXi8ZhU18i2uWLidRUotS, async-2.2.2-K8T9LglWxlG5HgD5vvGjbo, optparse-applicative-0.16.1.0-JDPEASK1GJS1Nsq2qjjZCq
installed package tasty-quickcheck-0.10.1.2 is broken due to missing package QuickCheck-2.14.2-io0WylueSG4seqSTUQCKM, optparse-applicative-0.16
Admin
Andreas Baumann commented on 26.01.2021 20:19

We should make sure first, that the Haskell updates from upstream are really applied to
our build schedule in exactly that order (as I fear order of rebuilds matters a lot).

As an easy fix in the past I was successful by selectively building parts of the failed Haskell module, but I easily ended up in dependency circles (where this library hash doesn't come in
handy at all!)

As I last resort I can think of a script checking the build error logs and trying to
guess package names which needs to be rebuilt:

Configuring attoparsec-0.13.2.4...
Error:
    The following packages are broken because other packages they depend on are missing. These broken packages must be rebuilt before they can be used.
installed package tasty-1.3.1 is broken due to missing package ansi-terminal-0.11-2SXi8ZhU18i2uWLidRUotS, async-2.2.2-K8T9LglWxlG5HgD5vvGjbo, optparse-applicative-0.16.1.0-JDPEASK1GJS1Nsq2qjjZCq
installed package tasty-quickcheck-0.10.1.2 is broken due to missing package QuickCheck-2.14.2-io0WylueSG4seqSTUQCKM, optparse-applicative-0.16.1.0-JDPEASK1GJS1Nsq2qjjZCq, random-1.2.0-1bl1BCDTJTv7mTz0mMuFeh

So in this example this means haskell-tasty needs a rebuild against new versions of
haskell-async, haskell-optparse-applicative and tasty-quickcheck one against haskell
quickcheck, haskell-optparse-applicative and haskell-random.

I also fear we cannot simply drop Haskell easilily for things like 'pandoc' or
'shellcheck'.

Admin
Erich Eckner commented on 31.01.2021 09:20

"As I last resort I can think of a script checking the build error logs and trying to
guess package names which needs to be rebuilt:"

We do that already: the buildmaster does reschedule the respective packages upon report of error. However, as you noted: Dependency cycles make this approach fail, too, because then, everything depending on haskell-tasty or haskell-tasty-quickchek which was already built needs to be rebuilt, too …

check the build scripts starting from:

https://git.archlinux32.org/builder/tree/bin/return-assignment#n291

maybe the regex does not work anymore or something like that …

Admin
Erich Eckner commented on 31.01.2021 09:38

nope, the regex looks good.

Admin
Erich Eckner commented on 31.01.2021 15:50

I had a (brief) look into the haskell mess and the problem seems to be, that a few haskell-* packages genuinely fail (e.g. haskell-tasty) on 32 bit.

The buildmaster ignores failed packages after some time (a few days, I think). The reasoning behind that is, that for "normal" packages (like libraries), this will not block all dependent packages (imagine libfoo and package bar, depending on libfoo. Both get an update, but libfoo fails to build. At some point, we want to have the new bar even if it's compiled against the old libfoo - this is especially true, if "libfoo" is actually just some boring makedependency, like git, or some doctohtml or whatnotelse). Unfortunately, with haskell-* this won't work, because the dependency *must* be rebuilt to update the hash (e.g. makedepends pandoc will not work in the old version).

Admin
Erich Eckner commented on 31.01.2021 16:34

These haskell-* packages have build errors which are not "due to haskell itself":

haskell-generic-data
haskell-libbf
haskell-profunctors
haskell-quickcheck-classes-base
haskell-rank2classes
haskell-semigroupoids
haskell-tasty
haskell-th-lift-instances
haskell-tidal

Admin
Andreas Baumann commented on 11.02.2021 05:15

Now ghc itself fails:

"inplace/bin/mkdirhier" libraries/ghc-prim/dist-install/doc/html/ghc-prim//.
"inplace/bin/mkdirhier" libraries/ghc-prim/dist-install/build/autogen/GHC//.
"inplace/bin/genprimopcode" --make-haskell-source < compiler/stage1/build/primops.txt > libraries/ghc-prim/dist-install/build/autogen/GHC/Prim.hs
"inplace/bin/ghc-cabal" hscolour libraries/ghc-prim dist-install
ghc-cabal: The program 'hscolour' version >=1.8 is required but the version of
/usr/bin/HsColour could not be determined.
make[1]: *** [libraries/ghc-prim/ghc.mk:4: libraries/ghc-prim/dist-install/doc/html/ghc-prim/ghc-prim.haddock] Error 1
make: *** [Makefile:128: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...
Admin
Andreas Baumann commented on 20.02.2021 10:25

and again on i686:

  1. > Compressing package…

zstd: error 11 : Allocation error : not enough memory

Admin
Erich Eckner commented on 06.03.2021 08:14

Considering the ridiculous speed, at which upstream increases haskell pkgrels, I believe, they have similar issues as we have. But for them, they are not as serious as for us, because they have a less strict dependency-checking in place than we do.

I added two TODOs:
https://archlinux32.org/buildmaster/todos.php#TODO232 https://archlinux32.org/buildmaster/todos.php#TODO233 but didn't have time yet to actually implement this. If implemented, they should considerably speed up the haskell rebuilds.

Admin
Andreas Baumann commented on 21.10.2021 05:14

In latest builds I see also 'ghc' itself choaking on 'llvm' for unknown reasons:

ghc: panic! (the 'impossible' happened)
  (GHC version 8.10.5:
        Failed to lookup LLVM data layout
  Target: i686-unknown-linux
  Available targets:
      i386-unknown-windows
      i686-unknown-windows
      x86_64-unknown-windows
      arm-unknown-linux-gnueabihf
      arm-unknown-linux-musleabihf
      armv6-unknown-linux-gnueabihf
      armv6-unknown-linux-musleabihf
      armv6l-unknown-linux-gnueabihf
      armv6l-unknown-linux-musleabihf
      armv7-unknown-linux-gnueabihf
      armv7-unknown-linux-musleabihf
      armv7a-unknown-linux-gnueabi
      armv7a-unknown-linux-musleabi
      armv7a-unknown-linux-gnueabihf
      armv7a-unknown-linux-musleabihf
      armv7l-unknown-linux-gnueabi
      armv7l-unknown-linux-musleabi
      armv7l-unknown-linux-gnueabihf
      armv7l-unknown-linux-musleabihf
      aarch64-unknown-linux-gnu
      aarch64-unknown-linux-musl
      aarch64-unknown-linux
      i386-unknown-linux-gnu
      i386-unknown-linux-musl
      i386-unknown-linux
      x86_64-unknown-linux-gnu
      x86_64-unknown-linux-musl
      x86_64-unknown-linux
      x86_64-unknown-linux-android
      armv7-unknown-linux-androideabi
      aarch64-unknown-linux-android
      armv7a-unknown-linux-androideabi
      powerpc64le-unknown-linux-gnu
      powerpc64le-unknown-linux-musl
      powerpc64le-unknown-linux
      s390x-ibm-linux
      i386-apple-darwin
      x86_64-apple-darwin
      arm64-apple-darwin
      aarch64-apple-darwin
      armv7-apple-ios
      aarch64-apple-ios
      i386-apple-ios
      x86_64-apple-ios
      amd64-portbld-freebsd
      x86_64-unknown-freebsd
      aarch64-unknown-freebsd
      armv6-unknown-freebsd-gnueabihf
      armv7-unknown-freebsd-gnueabihf
      arm-unknown-nto-qnx-eabi
  Call stack:
      CallStack (from HasCallStack):
        callStackDoc, called at compiler/utils/Outputable.hs:1179:37 in ghc:Outputable
        pprPanic, called at compiler/llvmGen/LlvmCodeGen.hs:113:20 in ghc:LlvmCodeGen

Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

ghc/ghc.mk:124: ghc/stage1/package-data.mk: No such file or directory
make[1]: *** [utils/ghc-cabal/ghc.mk:59: utils/ghc-cabal/dist/build/tmp/ghc-cabal] Error 1

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing