Bug 2554

I'm pretty sure 1 of the 4 jobs available been blocked didn't helped to build mame. Please resubmit your jobs (preferably at the end of the day).

For the record, we cannot put random memory into our i7 8Go server hosted by our Internet provider. Really, that's not going to make it that way.

Thank you for the clarification. To be clear, I was not planning on sending random memory sticks - I was thinking of buying the memory and setting the shipping address to wherever the server physically is.
0.147u2 is already built for rawhide, it took 11 hours [1]. In any case, before I submit builds for other branches, could you move 0.147 to stable on F-18? Looking at the timelines this should probably be the version in the release repo.

[1] http://buildsys.rpmfusion.org/build-status/job.psp?uid=14907

Building for F-17 only took 2 hours:
http://buildsys.rpmfusion.org/build-status/job.psp?uid=14966
This indicates that there might be something wrong with rawhide build stack.

F18 wins the prize for the longest build ever: i686 and x86_64 took 1250 and 1532 minutes, respectively. It makes me think something is really wrong with the F18+ gcc stack or something.

Here is one of the possible culprits:
Here's one difference between F18 and earlier that does affect memory usage:
https://fedoraproject.org/wiki/Features/DwarfCompressor
See https://bugzilla.redhat.com/show_bug.cgi?id=833311 and /etc/rpm/macros.dwz for further info on memory consumption and tunables.

Disabling dwz has brought the build time back to 2 hours, so it seems like it is the source of the issues we have been seeing. I'll try to look into how much memory would be needed to enable it back.

With just looking at the gnome-system-monitor, find-debuginfo.sh topped at 2.9 GB for the x86_64 build. No surprise two concurrent builds are bringing the builder to its knees.

I am afraid we need to look into this again. I have tried to build mame several times yesterday and was unable to succeed. The error was something about ld being killed. Granted, linking mame takes up upwards of 5 GiB of memory, so building i686 and x86_64 in parallel will require quite a lot of it.

0.163 failed as well (job 22680 @ i686):

collect2: error: ld terminated with signal 9 [Killed]
mame.make:223: recipe for target '../../../../../mame' failed
Makefile:898: recipe for target 'mame' failed
make[2]: *** [../../../../../mame] Error 1
make[1]: *** [mame] Error 2
make[1]: Leaving directory '/builddir/build/BUILD/mame-0.163/build/projects/sdl/mame/gmake-linux'
makefile:911: recipe for target 'linux_x86' failed
make: *** [linux_x86] Error 2

Could I close bugs #2992 and #2777  ?  seems fixed now .

I'm writing here because I notice that we don't have mame for F22 

RPMFusion nonfree for Fedora 22 don't have this packages: 

gnome-video-arcade.x86_64
mame.x86_64
mame-data.noarch
mame-data-extras.noarch
mame-data-extras-robby.noarch
mame-data-software-lists.noarch
mame-ldplayer.x86_64
mame-tools.x86_64


Neither in SRPMS 
http://download1.rpmfusion.org/nonfree/fedora/development/22/source/SRPMS/
or 
http://download1.rpmfusion.org/nonfree/fedora/releases/22/Everything/source/SRPMS/


Thanks.

The bugs you are referring to are obsolete and can be closed. Apologies for missing them.
Regarding lack of mame in F-22 - the reason is the same there is no updated mame for F-21 or similar. The way mame is build has been changed and the builders do not have enough memory to build x86_64 and i686 in parallel.

have you try not use -j4 ? 

-make %{?_smp_mflags} $MAME_FLAGS TARGET=ldplayer OPT_FLAGS="$RPM_OPT_FLAGS"
+make $MAME_FLAGS TARGET=ldplayer OPT_FLAGS="$RPM_OPT_FLAGS"

I just did, let's see if it helps.

As expected, removing smp_mflags did not help. Linking mame requires approximately GB of memory. Depending of how much is available on the builder, it might be possible to work the problem around by reducing the number of concurrent builds from 4 to 1.

OK , yes it put make less aggressive but not solve things on ld . 

Aside note :

can't compile in my Core i5 with 4 gb of ram, when : 

%bcond_without debug
%bcond_without simd

ends with : 
collect2: error: ld returned 1 exit status
mame.make:223: recipe for target '../../../../../mame64d' failed
Makefile:898: recipe for target 'mame' failed
make[1]: Leaving directory '/builddir/build/BUILD/mame-0.163/build/projects/sdl/mame/gmake-linux'
makefile:904: recipe for target 'linux_x64' failed
make[2]: *** [../../../../../mame64d] Error 1


and only with : 
%bcond_without debug

python src/build/file2str.py src/mame/layout/mdrawpkr.lay build/generated/mame/layout/mdrawpkr.lh layout_mdrawpkr
Unable to open output file 'build/generated/mame/layout/mdrawpkr.lh'

I will see if the ld switches mentioned by Dan back in 2012 make a big enough difference.

Hi, 
Not compiling at all in my Fedora 21 and for Fedora 21. with cvs devel version without modifications . 

Importing drivers from '../../../../../src/mame/mess.lst'
32301 drivers found
Compiling generated/mame/mame/drivlist.c...
g++    -MMD -MP -DPTR64=1 -DNDEBUG -DCRLF=2 -DLSB_FIRST -DUSE_SYSTEM_JPEGLIB -DUSE_SYSTEM_PORTMIDI -DUSE_SYSTEM_SQLITE -DNATIVE_DRC=drcbe_x64 -DLUA_COMPAT_APIINTCASTS -I../../../../../src/osd -I../../../../../src/emu -I../../../../../src/mame -I../../../../../src/lib -I../../../../../src/lib/util -I../../../../../3rdparty -I../../../../generated/mame/layout -I../../../../generated/resource  -g -m64 --pipe -g2 -fno-omit-frame-pointer -fno-optimize-sibling-calls -Wno-deprecated-declarations -O2 -fno-strict-aliasing -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -Wno-unknown-pragmas -Wall -Wcast-align -Wundef -Wformat-security -Wwrite-strings -Wno-sign-compare -Wno-conversion -Wno-unused-result -Wno-narrowing -Wno-attributes -Wno-array-bounds -m64 -DINLINE="static inline" -x c++ -std=gnu++98 -Woverloaded-virtual  -o "../../../../linux_gcc/obj/x64/Release/generated/mame/mame/drivlist.o" -MF ../../../../linux_gcc/obj/x64/Release/generated/mame/mame/drivlist.d -c "../../../../generated/mame/mame/drivlist.c"

Linking mame64...
g++ -o ../../../../../mame64 ../../../../linux_gcc/obj/x64/Release/src/mame/mame.o (...)

collect2: error: ld returned 1 exit status
mame.make:223: recipe for target '../../../../../mame64' failed
Makefile:898: recipe for target 'mame' failed
make[1]: Leaving directory '/builddir/build/BUILD/mame-0.163/build/projects/sdl/mame/gmake-linux'
makefile:904: recipe for target 'linux_x64' failed
make[2]: *** [../../../../../mame64] Error 1
make[1]: *** [mame] Error 2
make: *** [linux_x64] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.WGaXkK (%build)

This is strange, I built it in mock several times already.

In any case, with the workarounds the linker tops out at 2.6 GiB vs 5.5 GiB. I will be submitting another build shortly.

(In reply to comment #18)
> This is strange, I built it in mock several times already.

sorry I had remember now that was compile at first time , without -j4 as mention in comment #12, just took much more time, I 'm positive because I have the results here [1] , thinking if this is a not ccache bug ... 

[1]
-rw-rw-r-- 1 sergio mock  25653072 Jul  8 03:41 mame-0.163-1.fc21.x86_64.rpm
-rw-rw-r-- 1 sergio mock   1328980 Jul  8 03:41 mame-tools-0.163-1.fc21.x86_64.rpm
-rw-rw-r-- 1 sergio mock   1341448 Jul  8 03:41 mame-ldplayer-0.163-1.fc21.x86_64.rpm
-rw-rw-r-- 1 sergio mock     40064 Jul  8 03:41 mame-data-0.163-1.fc21.noarch.rpm
-rw-rw-r-- 1 sergio mock   8560920 Jul  8 03:41 mame-data-software-lists-0.163-1.fc21.noarch.rpm
-rw-rw-r-- 1 sergio mock 331853180 Jul  8 03:46 mame-debuginfo-0.163-1.fc21.x86_64.rpm

I'm still trying to follow this issue.
Few things in mind. Anyone tested the x86_32 bit mock build explicitly ?
We are using a 64bit kernel on 32bit target, but process might still be limited.

Which amount of memory would allow to build mame ?
- with the minmal memory usage ?
- with normal options to be expected for efficient build ?
Can you please answear the question for both 32bit and 64bit flavour ? (armhfp will have to stay in less than 2Gio until we can access aarch64 builders)

It turns out there was an error in 32 bit build which I have now fixed. For x86_64 the rough estimates are as follows:
- without any of the workarounds, linker reaches 5.5 GiB and dwz 6.5 GiB
- the lowmem workaround does the following: disables dwz and adds -Wl,--no-keep-memory,--reduce-memory-overheads linker flags. This reduces the linker requirements to 2.6 GiB and dwz to zero
The workarounds actually allowed the build to survive the linking stage, it failed later due to spec error.
I can try to check the requirements for i686 build later today.

(In reply to comment #20)
> (In reply to comment #18)
> > This is strange, I built it in mock several times already.

I found the problem, it was disk space , after free 21 GB on my hard disk , all builds were successful. So I think we are running in the same problem in builder, may I ask Xavier to free some space in builder ?

Nicolas: i686 build linker maxes out at 3.8 GiB with workarounds disabled.

The build still failed with extra linker flags. I am now trying with -g1 added as well. Locally it reduced the memory needed to link to 1 GiB

I went to IRC #rpmfusion-admin , ask to free space on builder1 ,  after talking a bit with kwizart , he wrote: sergiomb, I'm going to test a scratch build on the infra, so it has to pass before we dig further.

https://koji.kwizart.net/koji/packageinfo?packageID=213 
https://koji.kwizart.net/koji/buildinfo?buildID=1458

and mame fails to compile on armv7hl [1] ,
g++: error: unrecognized command line option '-m32' 
nothing with  memory allocation .

Anyway 


[1] http://kojihub.kwizart.net/kojifiles/work/tasks/9546/29546/build.log

Anyway commits like :
"Replaced wildcards with || approach" and "Corrected the wildcards used in %%install section" doesn't fix any memory issue, previous code was correct and working, so if it fails there , just supports my opinion , that is disk space .

The previous code was not correct and this is why these commits were added. I am normally only testing on x86_64 locally and the existing code was failing on i686. As the build was being killed at linking stage, it never made it to %%install so this went unnoticed.

Memory issue is worked around:
http://buildsys.rpmfusion.org/build-status/job.psp?uid=22680

For the arm build failure, I am afraid ExcludeArch might be needed for now as don't understand the mame build system to fix it yet.

(In reply to comment #29)
> Memory issue is worked around:
> http://buildsys.rpmfusion.org/build-status/job.psp?uid=22680
> 
> For the arm build failure, I am afraid ExcludeArch might be needed for now as
> don't understand the mame build system to fix it yet.

You can use the ExcludeArch: %{arm} at the condition to fill a bugreport against mame in this bugzilla and block #2407
Note that if you don't use ExcludeArch right now, the build will still fails for f22.

About the arm compiler flag issue, you cannot use -m32 on armv7 because it's implicit. It doesn't make sense to use a native armv7 compiler to produce something else than a 32bit code. I would "grep" the entire build scripts to search a wrong occurrence of -m32 incorrectly assumed.

Hi, 

(In reply to comment #21)
> I'm still trying to follow this issue.
> Few things in mind. Anyone tested the x86_32 bit mock build explicitly ?
> We are using a 64bit kernel on 32bit target, but process might still be
> limited.
> 
> Which amount of memory would allow to build mame ?
> - with the minmal memory usage ?
> - with normal options to be expected for efficient build ?
> Can you please answear the question for both 32bit and 64bit flavour ? (armhfp
> will have to stay in less than 2Gio until we can access aarch64 builders)

A few days ago, when I built mame, without lowmem, for Fedora 21 in my laptop, ld process [1] used 74,2% of RAM, 2.765 GB and I think more 3.7 of virtual memory, so more than 6 GB! 

[1] 
  PID VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                     
28672 3724000 2,765g 496 D   2,7 74,2   0:17.24 ld 

note: line copied from top command

Closing, since current builders can build mame.