Testing FORCE MAKE JOBS on FreeBSD 8.0 - Revision history

Tykling at 12:59, 17 April 2010

2010-04-17T12:59:38Z

← Older revision		Revision as of 12:59, 17 April 2010
Line 1:		Line 1:
			{{DISPLAYTITLE:Testing FORCE_MAKE_JOBS on FreeBSD 8.0}}
	This is a test of the difference in port building speed with and without <code>FORCE_MAKE_JOBS</code> in <code>/etc/make.conf</code>.		This is a test of the difference in port building speed with and without <code>FORCE_MAKE_JOBS</code> in <code>/etc/make.conf</code>.

Tykling: moved Force make jobs to Testing FORCE MAKE JOBS on FreeBSD 8.0

2010-04-17T12:58:38Z

moved Force make jobs to Testing FORCE MAKE JOBS on FreeBSD 8.0

← Older revision	Revision as of 12:58, 17 April 2010
(No difference)

Tykling: /* Introduction */

2010-04-17T12:58:13Z

Introduction

← Older revision		Revision as of 12:58, 17 April 2010
Line 4:		Line 4:
	Parallel ports building in FreeBSD was introduced in FreeBSD 8.0, quoting from [[http://lists.freebsd.org/pipermail/freebsd-ports/2009-March/053736.html the initial mailing list post]]:		Parallel ports building in FreeBSD was introduced in FreeBSD 8.0, quoting from [[http://lists.freebsd.org/pipermail/freebsd-ports/2009-March/053736.html the initial mailing list post]]:
	<pre>		<pre>
	Two days ago, I have checked in probably most requested feature of last few years. Ports framework now systematically supports building ports on multiple processing cores. It is achieved by passing -jX flag to make(1) running on vendor code. Of course not all ports handle this well, experimental run on pointyhat with this flag globally enabled turned up shy of 400 failures. Because of that, the feature was designed as a whitelist. Individual ports need to be enabled, and indeed, fellow developers took on and already started adding required declarations to popular ports like Firefox and others.		Two days ago, I have checked in probably most requested feature of last
			few years. Ports framework now systematically supports building ports on
			multiple processing cores. It is achieved by passing -jX flag to make(1)
			running on vendor code. Of course not all ports handle this well,
			experimental run on pointyhat with this flag globally enabled turned up
			shy of 400 failures. Because of that, the feature was designed as a
			whitelist. Individual ports need to be enabled, and indeed, fellow
			developers took on and already started adding required declarations to
			popular ports like Firefox and others.

	If you are FreeBSD ports user:		If you are FreeBSD ports user:

	You don't need to do anything to enable the new feature. Whitelisted ports will automatically make use of all processors available in your computer. If you want, for some reasons, to disable this feature, put DISABLE_MAKE_JOBS=yes to your /etc/make.conf. By default, the level of parallelization will be equal to a number of processing cores in your machine. If you want to override this number, use for example MAKE_JOBS_NUMBER=6, again in /etc/make.conf. And if you are extra brave, or you want to check out all the yet unmarked ports, if they will build, you can define FORCE_MAKE_JOBS=yes in /etc/make.conf.		You don't need to do anything to enable the new feature. Whitelisted
			ports will automatically make use of all processors available in your
			computer. If you want, for some reasons, to disable this feature, put
			DISABLE_MAKE_JOBS=yes to your /etc/make.conf. By default, the level of
			parallelization will be equal to a number of processing cores in your
			machine. If you want to override this number, use for example
			MAKE_JOBS_NUMBER=6, again in /etc/make.conf. And if you are extra brave,
			or you want to check out all the yet unmarked ports, if they will build,
			you can define FORCE_MAKE_JOBS=yes in /etc/make.conf.
	</pre>		</pre>

Tykling at 12:56, 17 April 2010

2010-04-17T12:56:40Z

← Older revision		Revision as of 12:56, 17 April 2010
Line 1:		Line 1:
	This is a test of the difference in port building speed with and without <code>FORCE_MAKE_JOBS</code> in <code>/etc/make.conf</code>.		This is a test of the difference in port building speed with and without <code>FORCE_MAKE_JOBS</code> in <code>/etc/make.conf</code>.

	== Introduction ==		== Introduction ==
			Parallel ports building in FreeBSD was introduced in FreeBSD 8.0, quoting from [[http://lists.freebsd.org/pipermail/freebsd-ports/2009-March/053736.html the initial mailing list post]]:
			<pre>
			Two days ago, I have checked in probably most requested feature of last few years. Ports framework now systematically supports building ports on multiple processing cores. It is achieved by passing -jX flag to make(1) running on vendor code. Of course not all ports handle this well, experimental run on pointyhat with this flag globally enabled turned up shy of 400 failures. Because of that, the feature was designed as a whitelist. Individual ports need to be enabled, and indeed, fellow developers took on and already started adding required declarations to popular ports like Firefox and others.

			If you are FreeBSD ports user:

			You don't need to do anything to enable the new feature. Whitelisted ports will automatically make use of all processors available in your computer. If you want, for some reasons, to disable this feature, put DISABLE_MAKE_JOBS=yes to your /etc/make.conf. By default, the level of parallelization will be equal to a number of processing cores in your machine. If you want to override this number, use for example MAKE_JOBS_NUMBER=6, again in /etc/make.conf. And if you are extra brave, or you want to check out all the yet unmarked ports, if they will build, you can define FORCE_MAKE_JOBS=yes in /etc/make.conf.
			</pre>

			== Test setup ==
	There is a whole lot of ports installed on this machine (X + applications like Firefox and OpenOffice):		There is a whole lot of ports installed on this machine (X + applications like Firefox and OpenOffice):
	<pre>		<pre>

Tykling at 12:25, 17 April 2010

2010-04-17T12:25:46Z

← Older revision		Revision as of 12:25, 17 April 2010
Line 20:		Line 20:
	</pre>		</pre>

	The machine is configured with ZFS and is running great with X, everything seems to work. I've posted a dmesg [here].		The machine is configured with ZFS and is running great with X, everything seems to work. I've posted a [[Dmesg_-_Lenovo_x301_FreeBSD_8.0-release_amd64\|dmesg here]].

	== Testing ==		== Testing ==

Tykling at 12:20, 17 April 2010

2010-04-17T12:20:04Z

← Older revision		Revision as of 12:20, 17 April 2010
Line 1:		Line 1:
	This is a test of the difference in port building speed with and without <code>FORCE_MAKE_JOBS</code> in <code>/etc/make.conf</code>. There is a whole lot of ports installed on this machine (X + applications like Firefox and OpenOffice):		This is a test of the difference in port building speed with and without <code>FORCE_MAKE_JOBS</code> in <code>/etc/make.conf</code>.

			== Introduction ==
			There is a whole lot of ports installed on this machine (X + applications like Firefox and OpenOffice):
	<pre>		<pre>
	[tykling@tyklappy ~]$ pkg_info \| wc -l		[tykling@tyklappy ~]$ pkg_info \| wc -l

Tykling at 12:19, 17 April 2010

2010-04-17T12:19:36Z

← Older revision		Revision as of 12:19, 17 April 2010
Line 1:		Line 1:
	A whole lot of ports installed:		This is a test of the difference in port building speed with and without <code>FORCE_MAKE_JOBS</code> in <code>/etc/make.conf</code>. There is a whole lot of ports installed on this machine (X + applications like Firefox and OpenOffice):
	<pre>		<pre>
	[tykling@tyklappy ~]$ pkg_info \| wc -l		[tykling@tyklappy ~]$ pkg_info \| wc -l
Line 5:		Line 5:
	</pre>		</pre>

			The machine is a Lenovo X301 with a solid-state disk and the following CPU and RAM:
			<pre>
			[tykling@tyklappy ~]$ grep "l memory" /var/run/dmesg.boot
			real memory = 6442450944 (6144 MB)
			avail memory = 6066290688 (5785 MB)
			</pre>

			<pre>
			[tykling@tyklappy ~]$ grep CPU: /var/run/dmesg.boot
			CPU: Intel(R) Core(TM)2 Duo CPU U9600 @ 1.60GHz (1602.39-MHz K8-class CPU)
			</pre>

			The machine is configured with ZFS and is running great with X, everything seems to work. I've posted a dmesg [here].

			== Testing ==
	This is the command I run to rebuild all ports (except for portmaster):		This is the command I run to rebuild all ports (except for portmaster):
	<pre>		<pre>
Line 10:		Line 25:
	</pre>		</pre>

	== Test 1 ==		=== Test 1 ===
	First test is a build with the default settings.		First test is a build with the default settings.

Line 27:		Line 42:
	</pre>		</pre>

	== Test 2 ==		=== Test 2 ===
	Second test is with <code>FORCE_MAKE_JOBS</code> enabled - <code>MAKE_JOBS_NUMBER</code> is set to 2 in this test, which would have been the default if I had not set <code>MAKE_JOBS_NUMBER</code> since there is two cores in the machine I am running this on.		Second test is with <code>FORCE_MAKE_JOBS</code> enabled - <code>MAKE_JOBS_NUMBER</code> is set to 2 in this test, which would have been the default if I had not set <code>MAKE_JOBS_NUMBER</code> since there is two cores in the machine I am running this on.

Line 38:		Line 53:
	PERL_VERSION=5.8.9		PERL_VERSION=5.8.9
	</pre>		</pre>



Line 49:		Line 63:


	== Test 3 ==		=== Test 3 ===
	Third test is with <code>FORCE_MAKE_JOBS</code> enabled - <code>MAKE_JOBS_NUMBER</code> is set to 4 in this test, double the number of cores in the machine.		Third test is with <code>FORCE_MAKE_JOBS</code> enabled - <code>MAKE_JOBS_NUMBER</code> is set to 4 in this test, double the number of cores in the machine.

Line 60:		Line 74:
	PERL_VERSION=5.8.9		PERL_VERSION=5.8.9
	</pre>		</pre>



Line 67:		Line 80:
	real 339m		real 339m
	</pre>		</pre>
	(~~from~~ memory, I don't have the rest of it)		(From memory, I don't have the rest of it)

			== Conclusion ==
			From 378 minutes to 343 minutes, or 35 minutes saved. Not bad.

Tykling at 12:01, 17 April 2010

2010-04-17T12:01:56Z

← Older revision		Revision as of 12:01, 17 April 2010
Line 39:		Line 39:
	</pre>		</pre>



			Resultat:
	<pre>		<pre>
	real 343m5.886s		real 343m5.886s
Line 58:		Line 61:
	</pre>		</pre>



			Resultat:
	<pre>		<pre>
	real 339m		real 339m
	</pre>		</pre>
	(from memory, I don't have the rest of it)		(from memory, I don't have the rest of it)

Tykling at 12:01, 17 April 2010

2010-04-17T12:01:03Z

← Older revision		Revision as of 12:01, 17 April 2010
Line 11:		Line 11:

	== Test 1 ==		== Test 1 ==
			First test is a build with the default settings.

	<code>/etc/make.conf</code>:		<code>/etc/make.conf</code>:
	<pre>		<pre>
Line 26:		Line 28:

	== Test 2 ==		== Test 2 ==
			Second test is with <code>FORCE_MAKE_JOBS</code> enabled - <code>MAKE_JOBS_NUMBER</code> is set to 2 in this test, which would have been the default if I had not set <code>MAKE_JOBS_NUMBER</code> since there is two cores in the machine I am running this on.

	<code>/etc/make.conf</code>:		<code>/etc/make.conf</code>:
	<pre>		<pre>
Line 42:		Line 46:


	== Test 2 ==		== Test 3 ==
			Third test is with <code>FORCE_MAKE_JOBS</code> enabled - <code>MAKE_JOBS_NUMBER</code> is set to 4 in this test, double the number of cores in the machine.

	<code>/etc/make.conf</code>:		<code>/etc/make.conf</code>:
	<pre>		<pre>
Line 55:		Line 61:
	real 339m		real 339m
	</pre>		</pre>
			(from memory, I don't have the rest of it)

Tykling at 11:55, 17 April 2010

2010-04-17T11:55:12Z

@@ Line 1: / Line 1: @@
- portmaster --no-confirm -afB
+A whole lot of ports installed:
-Uden FORCE_MAKE_JOBS:
+This is the command I run to rebuild all ports (except for portmaster):
- real    378m53.008s
+== Test 1 ==
- user    403m40.086s
+<code>/etc/make.conf</code>:
- sys     99m14.101s
+<pre>
- real    343m5.886s
+Resultat:
- user    405m43.557s
+<pre>
- sys     98m56.720s
+real    378m53.008s
-FORCE_MAKE_JOBS med 4 jobs:
+== Test 2 ==
- real    339m
+# added by use.perl 2010-04-17 03:07:25