• Status New
  • Percent Complete
    0%
  • Task Type Bug Report
  • Category Packages → Packages: Build-list
  • Assigned To No-one
  • Operating System i686
  • Severity Low
  • Priority Medium
  • Reported Version
  • Due in Version Undecided
  • Due Date Undecided
  • Votes
  • Private
Attached to Project: Archlinux32
Opened by Andreas Baumann - 16.06.2018
Last edited by Andreas Baumann - 16.06.2018

FS#43 - qt5-webengine, firefox, firefox-developer-edition use too much memory

In file included from gen/third_party/WebKit/public/platform/modules/presentation/presentation.mojom-shared.h:24,
                 from gen/third_party/WebKit/public/platform/modules/presentation/presentation.mojom.h:37,
                 from ../../../../qtwebengine-everywhere-src-5.11.0/src/3rdparty/chromium/content/browser/frame_host/render_frame_host_impl.h:66,
                 from ../../../../qtwebengine-everywhere-src-5.11.0/src/3rdparty/chromium/content/browser/frame_host/frame_tree_node.h:18,
                 from ../../../../qtwebengine-everywhere-src-5.11.0/src/3rdparty/chromium/content/browser/devtools/browser_devtools_agent_host.cc:21:
gen/third_party/WebKit/public/platform/modules/presentation/presentation.mojom-shared-internal.h:139:35: warning: alignment 1 of ‘blink::mojom::internal::PresentationConnectionMessage_Data’ is less than 8 [-Wpacked-not-aligned]
 class MOJOM_SHARED_CONTENT_EXPORT PresentationConnectionMessage_Data {
                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
virtual memory exhausted: Cannot allocate memory
ninja: build stopped: subcommand failed.

Also in firefox:

91:37.06 ../../build/unix/gold/ld: fatal error: libxul.so: mmap: failed to allocate 1703955732 bytes for output file: Cannot allocate memory
91:37.06 collect2: error: ld returned 1 exit status
91:37.06 make[4]: *** [/build/firefox/src/mozilla-unified/config/rules.mk:701: libxul.so] Error 1
91:37.06 make[3]: *** [/build/firefox/src/mozilla-unified/config/recurse.mk:73: toolkit/library/target] Error 2
91:37.06 make[2]: *** [/build/firefox/src/mozilla-unified/config/recurse.mk:33: compile] Error 2
91:37.06 make[1]: *** [/build/firefox/src/mozilla-unified/config/rules.mk:434: default] Error 2
91:37.06 make: *** [client.mk:168: build] Error 2
Admin
Andreas Baumann commented on 16.06.2018 05:33

Build slaves should have a 4GB swap space (if they are virtual machines),
and sysctl vm.mmap_min_addr=0 should be set.

For containers I don't know what's best becauste systemd-nspawn has
a mind of its own. sysctl vm.mmap_min_addr=0 on the host helped here too.

Admin
Andreas Baumann commented on 16.06.2018 05:37

Another solution could be not to use the gold-ld.

Admin
Andreas Baumann commented on 16.06.2018 06:26

and another one:

CodeCache::InnerPointerToCodeCacheEntry’; use assignment or value-initialization instead [-Wclass-memaccess]
     memset(&cache_[0], 0, sizeof(cache_));
                                         ^
../../../../qtwebengine-everywhere-src-5.11.0/src/3rdparty/chromium/v8/src/frames.h:36:10: note: ‘struct v8::internal::InnerPointerToCodeCache::InnerPointerToCodeCacheEntry’ declared here
   struct InnerPointerToCodeCacheEntry {
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
{standard input}: Assembler messages:
{standard input}:15117: Warning: end of file not at end of a line; newline inserted
{standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
g++: fatal error: Terminated signal terminated program cc1plus
compilation terminated.
ninja: build stopped: subcommand failed.

This looks like truncated assembly to me. Maybe using tmpfile instead of -pipe?

Admin
Andreas Baumann commented on 17.06.2018 06:24

qt5-webkit fails now much later in a build race in a plugin. removing -pipe could
also help for all other packages running out of virtual memory, so maybe changing
the global build options to '-j1' and not '-pipe' in makepkg.conf is an idea.
Or we patch the affected packages only, but patching away a -pipe might not be as
easy as one may think.

Admin
Andreas Baumann commented on 20.06.2018 05:01

Still trouble with firefox and firefox-developer-edition, -pipe gets added
somewhere even if I change it in the build chroot configuration in makepkg.conf..

Admin
Andreas Baumann commented on 21.06.2018 11:46

-pipe doens't have a huge impact.

So I'll try with some special LDFLAGS -Wl,–no-keep-memory, after a hint in:

https://bugzilla.mozilla.org/show_bug.cgi?id=854535

Admin
Andreas Baumann commented on 23.06.2018 07:56

A top on a 64-bit Archlinux shows me the following during a build:

26963 arch      20   0 6336364   1.8g   1468 R   7.0  91.8   1:44.14 dump_syms

0 S arch     26958 26956  0  80   0 - 25355 -      20:29 pts/0    00:00:00 /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/_virtualenv/bin/python /data/firefox/src/mozilla-unified/toolkit/crashreporter/tools/symbolstore.py -c --vcs-info --install-manifest=/data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/_build_manifests/install/dist_include,/data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include -s /data/firefox/src/mozilla-unified /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/host/bin/dump_syms /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/crashreporter-symbols /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/toolkit/library/libxul.so
0 D arch     26963 26958  7  80   0 - 1146305 -    20:29 pts/0    00:00:34 /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/host/bin/dump_syms /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/toolkit/library/libxul.so

So, this dump_syms program will never work in an 32-bit address room. The question
is, can it be tuned or hacked? The question, why did it work till now and what
changed so it doesn't work currently?

Admin
Andreas Baumann commented on 02.07.2018 15:58

Another try using the standard linker instead of the gold one.

Admin
Andreas Baumann commented on 02.07.2018 19:43

I think, I'm in the wrong movie:

36:53.02 libxul.so
37:13.26 /usr/bin/ld: out of memory allocating 1000 bytes after a total of 898674688 bytes
37:13.26 collect2: error: ld returned 1 exit status

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing