• Status Closed
  • Percent Complete
    100%
  • Task Type Bug Report
  • Category Packages → Packages: Build-list
  • Assigned To No-one
  • Operating System i686
  • Severity Low
  • Priority Medium
  • Reported Version
  • Due in Version Undecided
  • Due Date Undecided
  • Votes
  • Private
Attached to Project: Arch Linux 32
Opened by Admin - 14.11.2019
Last edited by Andreas Baumann - 21.04.2020

FS#43 - qt5-webengine, firefox, firefox-developer-edition use too much memory

Attached to Project: Archlinux32
Opened by Andreas Baumann - 16.06.2018
Last edited by Andreas Baumann - 02.10.2019
 FS#43  - qt5-webengine, firefox, firefox-developer-edition use too much memory

In file included from gen/third_party/WebKit/public/platform/modules/presentation/presentation.mojom-shared.h:24,

               from gen/third_party/WebKit/public/platform/modules/presentation/presentation.mojom.h:37,
               from ../../../../qtwebengine-everywhere-src-5.11.0/src/3rdparty/chromium/content/browser/frame_host/render_frame_host_impl.h:66,
               from ../../../../qtwebengine-everywhere-src-5.11.0/src/3rdparty/chromium/content/browser/frame_host/frame_tree_node.h:18,
               from ../../../../qtwebengine-everywhere-src-5.11.0/src/3rdparty/chromium/content/browser/devtools/browser_devtools_agent_host.cc:21:

gen/third_party/WebKit/public/platform/modules/presentation/presentation.mojom-shared-internal.h:139:35: warning: alignment 1 of ‘blink::mojom::internal::PresentationConnectionMessage_Data’ is less than 8 [-Wpacked-not-aligned]
class MOJOM_SHARED_CONTENT_EXPORT PresentationConnectionMessage_Data {

                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

virtual memory exhausted: Cannot allocate memory
ninja: build stopped: subcommand failed.

Also in firefox:

91:37.06 ../../build/unix/gold/ld: fatal error: libxul.so: mmap: failed to allocate 1703955732 bytes for output file: Cannot allocate memory
91:37.06 collect2: error: ld returned 1 exit status
91:37.06 make[4]: * [/build/firefox/src/mozilla-unified/config/rules.mk:701: libxul.so] Error 1
91:37.06 make[3]:
* [/build/firefox/src/mozilla-unified/config/recurse.mk:73: toolkit/library/target] Error 2
91:37.06 make[2]: * [/build/firefox/src/mozilla-unified/config/recurse.mk:33: compile] Error 2
91:37.06 make[1]:
* [/build/firefox/src/mozilla-unified/config/rules.mk:434: default] Error 2
91:37.06 make: *** [client.mk:168: build] Error 2

Closed by Andreas Baumann
02.10.2019 19:26
Reason for closing: Fixed
Additional comments about closing:

Currently firefox and qt5-webengine build.
firefox-developer-edition is blacklisted as it causes too much trouble
already as the official released version, so we don’t to beta testing
here..

  Comments (10)
  Related Tasks (0/0)

Admin
Andreas Baumann commented on 16.06.2018 05:33

Build slaves should have a 4GB swap space (if they are virtual machines),
and sysctl vm.mmap_min_addr=0 should be set.

For containers I don’t know what’s best becauste systemd-nspawn has
a mind of its own. sysctl vm.mmap_min_addr=0 on the host helped here too.
Admin
Andreas Baumann commented on 16.06.2018 05:37

Another solution could be not to use the gold-ld.
Admin
Andreas Baumann commented on 16.06.2018 06:26

and another one:

CodeCache::InnerPointerToCodeCacheEntry’; use assignment or value-initialization instead [-Wclass-memaccess]

   memset(&cache_[0], 0, sizeof(cache_));
                                       ^

../../../../qtwebengine-everywhere-src-5.11.0/src/3rdparty/chromium/v8/src/frames.h:36:10: note: ‘struct v8::internal::InnerPointerToCodeCache::InnerPointerToCodeCacheEntry’ declared here

 struct InnerPointerToCodeCacheEntry {
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

{standard input}: Assembler messages:
{standard input}:15117: Warning: end of file not at end of a line; newline inserted
{standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
g++: fatal error: Terminated signal terminated program cc1plus
compilation terminated.
ninja: build stopped: subcommand failed.

This looks like truncated assembly to me. Maybe using tmpfile instead of -pipe?
Admin
Andreas Baumann commented on 17.06.2018 06:24

qt5-webkit fails now much later in a build race in a plugin. removing -pipe could
also help for all other packages running out of virtual memory, so maybe changing
the global build options to ‘-j1’ and not ‘-pipe’ in makepkg.conf is an idea.
Or we patch the affected packages only, but patching away a -pipe might not be as
easy as one may think.
Admin
Andreas Baumann commented on 20.06.2018 05:01

Still trouble with firefox and firefox-developer-edition, -pipe gets added
somewhere even if I change it in the build chroot configuration in makepkg.conf..
Admin
Andreas Baumann commented on 21.06.2018 11:46

-pipe doens’t have a huge impact.

So I’ll try with some special LDFLAGS -Wl,–no-keep-memory, after a hint in:

https://bugzilla.mozilla.org/show_bug.cgi?id=854535 Admin
Andreas Baumann commented on 23.06.2018 07:56

A top on a 64-bit Archlinux shows me the following during a build:

26963 arch 20 0 6336364 1.8g 1468 R 7.0 91.8 1:44.14 dump_syms

0 S arch 26958 26956 0 80 0 - 25355 - 20:29 pts/0 00:00:00 /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/_virtualenv/bin/python /data/firefox/src/mozilla-unified/toolkit/crashreporter/tools/symbolstore.py -c –vcs-info –install-manifest=/data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/_build_manifests/install/dist_include,/data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/include -s /data/firefox/src/mozilla-unified /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/host/bin/dump_syms /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/crashreporter-symbols /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/toolkit/library/libxul.so
0 D arch 26963 26958 7 80 0 - 1146305 - 20:29 pts/0 00:00:34 /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/dist/host/bin/dump_syms /data/firefox/src/mozilla-unified/obj-x86_64-pc-linux-gnu/toolkit/library/libxul.so

So, this dump_syms program will never work in an 32-bit address room. The question
is, can it be tuned or hacked? The question, why did it work till now and what
changed so it doesn’t work currently?
Admin
Andreas Baumann commented on 02.07.2018 15:58

Another try using the standard linker instead of the gold one.
Admin
Andreas Baumann commented on 02.07.2018 19:43

I think, I’m in the wrong movie:

36:53.02 libxul.so
37:13.26 /usr/bin/ld: out of memory allocating 1000 bytes after a total of 898674688 bytes
37:13.26 collect2: error: ld returned 1 exit status

Admin
Andreas Baumann commented on 02.10.2019 19:24

Rust things also run out of memory (firefox), disabling some debug info
with debug_info=1 makes the builds succeed.

Google Cache

Closed by  Andreas Baumann
21.04.2020 18:40
Reason for closing:  Works for me
Additional comments about closing:  

Current versions work with all kind of tricks like dropping debug info, not building
firefox just for optimizing some code afterwards, etc.

Loading...

Available keyboard shortcuts

Tasklist

Task Details

Task Editing