Does compiling from source interfere with the package manager?

[Toby Newman]

I've just compiled gamin from source to solve a memory leak problem it
was giving me. Having done this, effectively 'going behind the back of'
my package manager (adept), will the package manager cope okay once a
newer version of gamin becomes available from the repositories?

i.e. How clever is the package manager?

--
Toby.
Add the word afiduluminag to the subject
field to circumvent my email filters.
Ignore any mail delivery error.

[Jeffery Michael Thompson]

Toby Newman wrote:

> I've just compiled gamin from source to solve a memory leak problem it
> was giving me. Having done this, effectively 'going behind the back of'
> my package manager (adept), will the package manager cope okay once a
> newer version of gamin becomes available from the repositories?
>
> i.e. How clever is the package manager?
>

Not that clever.

Jeff

[Moe Trin]

On Wed, 4 Jan 2006, in the Usenet newsgroup alt.os.linux.ubuntu, in article
<slrndrn4aq.1vf.google@ID-171443.user.uni-berlin.de>, Toby Newman wrote:

>I've just compiled gamin from source to solve a memory leak problem it
>was giving me.

Have you also submitted a bug report?

>Having done this, effectively 'going behind the back of' my package
>manager (adept), will the package manager cope okay once a newer
>version of gamin becomes available from the repositories?

and the answer is "that depends". When you install the tarball, does
it ONLY effect the exact same files as the package? In that case,
there _probably_ won't be a problem updating _this_ package. Are there
other packages that depend on this package having a wart located in
this or that particular location? You may run into a problem, especially
if this is a (Debian or Ubuntu) patched ("improved") version of a package.
Other (Debian or Ubuntu) packages that depend on this application are
expecting the (Debian or Ubuntu) supplied package, and not a modified or
altered version.

>i.e. How clever is the package manager?

In some areas, very clever. In others - not clever at all. When updating
or when installing other packages, the package manager does not look at
the installed files, but consults a package database. If what is installed
differs from the database, the difference won't be noticed. It may not be
important, or it might be a really really big iceberg directly ahead of
your 'Titanic'.

Old guy

[Toby Newman]

On 2006-01-05, Moe Trin <ibuprofin@painkiller.example.tld> wrote:

> On Wed, 4 Jan 2006, in the Usenet newsgroup alt.os.linux.ubuntu, in article
><slrndrn4aq.1vf.google@ID-171443.user.uni-berlin.de>, Toby Newman wrote:
>

>>I've just compiled gamin from source to solve a memory leak problem it
>>was giving me.

>
> Have you also submitted a bug report?

No, the bug is already in the database, but it is marked 'resolved'. I'm
going to watch my system for a few days to learn about when it occurs
before I re-open the bug. At the moment, it seems to be fine unless the
2nd user on my system loads up Amarok. I'm not 100% sure, yet, so I
don't want to post a half-cut message :)

>>Having done this, effectively 'going behind the back of' my package
>>manager (adept), will the package manager cope okay once a newer
>>version of gamin becomes available from the repositories?

>
> and the answer is "that depends". When you install the tarball, does
> it ONLY effect the exact same files as the package?

I'm afraid I have no idea how to tell which files are affected by
#make install
or if any of those files are part of the package.

> In that case,
> there _probably_ won't be a problem updating _this_ package. Are there
> other packages that depend on this package having a wart located in
> this or that particular location? You may run into a problem, especially
> if this is a (Debian or Ubuntu) patched ("improved") version of a package.
> Other (Debian or Ubuntu) packages that depend on this application are
> expecting the (Debian or Ubuntu) supplied package, and not a modified or
> altered version.

What is a wart? I googled for it to no avail.

>>i.e. How clever is the package manager?

>
> In some areas, very clever. In others - not clever at all. When updating
> or when installing other packages, the package manager does not look at
> the installed files, but consults a package database. If what is installed
> differs from the database, the difference won't be noticed. It may not be
> important, or it might be a really really big iceberg directly ahead of
> your 'Titanic'.

Well you've well and truly put the sh!ts up me here. Since compiling and
installing gamin from source doesn't seem to have solved my problem,
should I just leave well alone, do a '# make uninstall', or
remove/reinstall via my package manager?

I really don't want to hose my system.

I thought installing from source was a good technique? Isn't that why I
learnt how to do it? Latest software, optimised for my machine... I
often get advised to do it.

--
Toby.
Add the word afiduluminag to the subject
field to circumvent my email filters.
Ignore any mail delivery error.

[Moe Trin]

On Thu, 5 Jan 2006, in the Usenet newsgroup alt.os.linux.ubuntu, in article
<slrndrr2n5.9dd.google@ID-171443.user.uni-berlin.de>, Toby Newman wrote:

> Moe Trin <ibuprofin@painkiller.example.tld> wrote:

>I'm afraid I have no idea how to tell which files are affected by
>#make install

In the source directory where you ran the 'make' command, there is a
'Makefile' which is a script that 'make' is using. Depending, the
install target will be using 'cp' or 'install' or 'mv' to take
specific files from the source and put them somewhere. Reading that
section will tell you which ones.

>or if any of those files are part of the package.

One would hope they are. 'dpkg' is the Debian package tool, and you
can use that to see what files are supposed to be in a package. (I'm
less familiar with dpkg than I should be, but it looks like

dpkg -L package_name

should list the files within that package.)

>> In that case, there _probably_ won't be a problem updating _this_
>> package. Are there other packages that depend on this package having
>> a wart located in this or that particular location? You may run into
>> a problem, especially if this is a (Debian or Ubuntu) patched
>> ("improved") version of a package. Other (Debian or Ubuntu) packages
>> that depend on this application are expecting the (Debian or Ubuntu)
>> supplied package, and not a modified or altered version.

>
>What is a wart? I googled for it to no avail.

:wart: n.
A small, {crock}y {feature} that sticks out of an otherwise {clean}
design. Something conspicuous for localized ugliness, especially a
special-case exception to a general rule. For example, in some
versions of csh(1), single quotes literalize every character inside
them except !. In ANSI C, the ?? syntax used for obtaining ASCII
characters in a foreign environment is a wart. See also {miswart}.

That's from the Jargon file. What I was trying to say is that packages
built for a specific distribution are often "improved" by the distributor
in one way or another. These specific improvements may add a feature, or
alter the way a feature operates. The problem is that the original author
often does not include these "improvements", or may implement them in a
different way. _Other_ packages from this distributor may depend on
this modification, and may not be compatible with the original application,
or packages made by other distributors. With a Debian package, there may be
a 'diff' file, which talks about changes it makes or it may be in the
Changelog(s).

>> If what is installed differs from the database, the difference won't be
>> noticed. It may not be important, or it might be a really really big
>> iceberg directly ahead of your 'Titanic'.

>
>Well you've well and truly put the sh!ts up me here.

Sorry - a lot depends on how much Debian or Ubuntu changed the original,
AND how much other packages may depend on this one. 'gamin' is a file
monitoring tool - I wouldn't expect that many dependencies.

>Since compiling and installing gamin from source doesn't seem to have
>solved my problem, should I just leave well alone, do a '# make uninstall',
>or remove/reinstall via my package manager?

This depends on how much the package differs from what the tarball installed,
and the only way to find that is to compare the dpkg listing with what the
makefile is doing. Likely, there isn't to much difference in what files
are effected, so using the package manager would be the preferred method.

>I really don't want to hose my system.
>
>I thought installing from source was a good technique? Isn't that why I
>learnt how to do it? Latest software, optimised for my machine... I
>often get advised to do it.

On a system with a package manager, whether apt, dpkg, emerge, pkgadd, or
RPM, it's not always the best idea. The package managers work best when they
have sole control over what is added/removed/altered on a system. This way,
they know what's going on and don't get surprised. Adding (or worse, replacing)
an application from source is like someone sneaking into your house when
you aren't looking, and replacing something. You may not notice until you
try to do something that the packaged version will do, but the other version
either doesn't do, or does differently.

On the other hand, let's say you want to add some application that your
distribution doesn't have. In that case, you have little choice but to
install from source. The concern is that (probably later) your distribution
creates a package for that application, and then goes on and creates some
other application that depends on this one, and the other application does
something you can't live without. You decide you want to install that, but
the package manager says 'whoa, you need to install this first package before
you can install the second". And you think "wait a minute - I've already
installed that". Well, you did, but the package manager doesn't know that.
It doesn't look for this or that file, it looks in it's own database to see
if _it_ had installed the package.

There are (literally) hundreds of Linux distributions. Some are derivatives
of others (Ubuntu is based on Debian, while the Brazilian "Console Linux"
is based on Red Hat), but each developer seems to think they know what is
best. This results in incompatibility, especially when there are dependencies
involved. Want the latest kernel?. Fine - that's 2.6.15, which was released
by kernel.org a little over three days ago. But on your system, if you run
the command 'uname -a' I'm sure you'll find that you aren't using a "stock"
Linux kernel - you'll see (if you're up to date on 5.10) 2.6.12-10.25. Only
problem is - that's not a Linux kernel numbering scheme

46709978 Jun 17 22:12 linux-2.6.12.tar.gz
46718915 Jun 22 22:00 linux-2.6.12.1.tar.gz
46714732 Jun 30 00:11 linux-2.6.12.2.tar.gz
46713120 Jul 15 21:57 linux-2.6.12.3.tar.gz
46714222 Aug 5 21:11 linux-2.6.12.4.tar.gz
46715219 Aug 15 00:43 linux-2.6.12.5.tar.gz
47996682 Aug 29 00:02 linux-2.6.13.tar.gz
46719369 Aug 29 17:18 linux-2.6.12.6.tar.gz

Those are the source tarballs on ftp.kernel.org. Notice - no dash numbers.
The -10.25 sequence is the Ubuntu modification to the kernel. You'd have to
read the Ubuntu documentation to see what changes have been made from the
stock kernel. The same is true for virtually every other package on every
other distribution.

The above listing also illustrates the problem of "Latest software". Playing
version number tag may mean you get to do a lot of compiling. Reading the
ChangeLog files will often give you the answer of "do I need this".

1048541 Jun 17 22:12 ChangeLog-2.6.12
1084 Jun 22 22:00 ChangeLog-2.6.12.1
5454 Jun 30 00:11 ChangeLog-2.6.12.2
7824 Jul 15 21:38 ChangeLog-2.6.12.3
11112 Aug 5 21:11 ChangeLog-2.6.12.4
7884 Aug 15 00:36 ChangeLog-2.6.12.5
2365137 Aug 29 02:58 ChangeLog-2.6.13
4614 Aug 29 17:18 ChangeLog-2.6.12.6

The x.y.x.N number sequence is new (there was a 2.6.8.1 in October 2004, but
this really didn't start until 2.6.11.1 last March), and is supposed to be
bug fixes to a x.y.x kernel. In fact, the 2.0.x, 2.2.x, and 2.4.x kernels
(all still supported) don't have this. The sensible idea is to look at
the change log, and decide if you need the new kernel/package/what-ever. Is
it a security update? Bug fix? Some new feature you can't live without? Or is
it something else you don't need? Only you can tell for sure.

Old guy

[Toby Newman]

On 2006-01-07, Moe Trin <ibuprofin@painkiller.example.tld> wrote:

> On Thu, 5 Jan 2006, in the Usenet newsgroup alt.os.linux.ubuntu, in article
><slrndrr2n5.9dd.google@ID-171443.user.uni-berlin.de>, Toby Newman wrote:
>

>> Moe Trin <ibuprofin@painkiller.example.tld> wrote:

>

>>I'm afraid I have no idea how to tell which files are affected by
>>#make install

>
> In the source directory where you ran the 'make' command, there is a
> 'Makefile' which is a script that 'make' is using. Depending, the
> install target will be using 'cp' or 'install' or 'mv' to take
> specific files from the source and put them somewhere. Reading that
> section will tell you which ones.
>

>>or if any of those files are part of the package.

>
> One would hope they are. 'dpkg' is the Debian package tool, and you
> can use that to see what files are supposed to be in a package. (I'm
> less familiar with dpkg than I should be, but it looks like
>
> dpkg -L package_name
>
> should list the files within that package.)
>

>>> In that case, there _probably_ won't be a problem updating _this_
>>> package. Are there other packages that depend on this package having
>>> a wart located in this or that particular location? You may run into
>>> a problem, especially if this is a (Debian or Ubuntu) patched
>>> ("improved") version of a package. Other (Debian or Ubuntu) packages
>>> that depend on this application are expecting the (Debian or Ubuntu)
>>> supplied package, and not a modified or altered version.

>>
>>What is a wart? I googled for it to no avail.

>
> :wart: n.
> A small, {crock}y {feature} that sticks out of an otherwise {clean}
> design. Something conspicuous for localized ugliness, especially a
> special-case exception to a general rule. For example, in some
> versions of csh(1), single quotes literalize every character inside
> them except !. In ANSI C, the ?? syntax used for obtaining ASCII
> characters in a foreign environment is a wart. See also {miswart}.
>
> That's from the Jargon file. What I was trying to say is that packages
> built for a specific distribution are often "improved" by the distributor
> in one way or another. These specific improvements may add a feature, or
> alter the way a feature operates. The problem is that the original author
> often does not include these "improvements", or may implement them in a
> different way. _Other_ packages from this distributor may depend on
> this modification, and may not be compatible with the original application,
> or packages made by other distributors. With a Debian package, there may be
> a 'diff' file, which talks about changes it makes or it may be in the
> Changelog(s).
>

>>> If what is installed differs from the database, the difference won't be
>>> noticed. It may not be important, or it might be a really really big
>>> iceberg directly ahead of your 'Titanic'.

>>
>>Well you've well and truly put the sh!ts up me here.

>
> Sorry - a lot depends on how much Debian or Ubuntu changed the original,
> AND how much other packages may depend on this one. 'gamin' is a file
> monitoring tool - I wouldn't expect that many dependencies.
>

>>Since compiling and installing gamin from source doesn't seem to have
>>solved my problem, should I just leave well alone, do a '# make uninstall',
>>or remove/reinstall via my package manager?

>
> This depends on how much the package differs from what the tarball installed,
> and the only way to find that is to compare the dpkg listing with what the
> makefile is doing. Likely, there isn't to much difference in what files
> are effected, so using the package manager would be the preferred method.
>

>>I really don't want to hose my system.
>>
>>I thought installing from source was a good technique? Isn't that why I
>>learnt how to do it? Latest software, optimised for my machine... I
>>often get advised to do it.

>
> On a system with a package manager, whether apt, dpkg, emerge, pkgadd, or
> RPM, it's not always the best idea. The package managers work best when they
> have sole control over what is added/removed/altered on a system. This way,
> they know what's going on and don't get surprised. Adding (or worse, replacing)
> an application from source is like someone sneaking into your house when
> you aren't looking, and replacing something. You may not notice until you
> try to do something that the packaged version will do, but the other version
> either doesn't do, or does differently.
>
> On the other hand, let's say you want to add some application that your
> distribution doesn't have. In that case, you have little choice but to
> install from source. The concern is that (probably later) your distribution
> creates a package for that application, and then goes on and creates some
> other application that depends on this one, and the other application does
> something you can't live without. You decide you want to install that, but
> the package manager says 'whoa, you need to install this first package before
> you can install the second". And you think "wait a minute - I've already
> installed that". Well, you did, but the package manager doesn't know that.
> It doesn't look for this or that file, it looks in it's own database to see
> if _it_ had installed the package.
>
> There are (literally) hundreds of Linux distributions. Some are derivatives
> of others (Ubuntu is based on Debian, while the Brazilian "Console Linux"
> is based on Red Hat), but each developer seems to think they know what is
> best. This results in incompatibility, especially when there are dependencies
> involved. Want the latest kernel?. Fine - that's 2.6.15, which was released
> by kernel.org a little over three days ago. But on your system, if you run
> the command 'uname -a' I'm sure you'll find that you aren't using a "stock"
> Linux kernel - you'll see (if you're up to date on 5.10) 2.6.12-10.25. Only
> problem is - that's not a Linux kernel numbering scheme
>
> 46709978 Jun 17 22:12 linux-2.6.12.tar.gz
> 46718915 Jun 22 22:00 linux-2.6.12.1.tar.gz
> 46714732 Jun 30 00:11 linux-2.6.12.2.tar.gz
> 46713120 Jul 15 21:57 linux-2.6.12.3.tar.gz
> 46714222 Aug 5 21:11 linux-2.6.12.4.tar.gz
> 46715219 Aug 15 00:43 linux-2.6.12.5.tar.gz
> 47996682 Aug 29 00:02 linux-2.6.13.tar.gz
> 46719369 Aug 29 17:18 linux-2.6.12.6.tar.gz
>
> Those are the source tarballs on ftp.kernel.org. Notice - no dash numbers.
> The -10.25 sequence is the Ubuntu modification to the kernel. You'd have to
> read the Ubuntu documentation to see what changes have been made from the
> stock kernel. The same is true for virtually every other package on every
> other distribution.
>
> The above listing also illustrates the problem of "Latest software". Playing
> version number tag may mean you get to do a lot of compiling. Reading the
> ChangeLog files will often give you the answer of "do I need this".
>
> 1048541 Jun 17 22:12 ChangeLog-2.6.12
> 1084 Jun 22 22:00 ChangeLog-2.6.12.1
> 5454 Jun 30 00:11 ChangeLog-2.6.12.2
> 7824 Jul 15 21:38 ChangeLog-2.6.12.3
> 11112 Aug 5 21:11 ChangeLog-2.6.12.4
> 7884 Aug 15 00:36 ChangeLog-2.6.12.5
> 2365137 Aug 29 02:58 ChangeLog-2.6.13
> 4614 Aug 29 17:18 ChangeLog-2.6.12.6
>
> The x.y.x.N number sequence is new (there was a 2.6.8.1 in October 2004, but
> this really didn't start until 2.6.11.1 last March), and is supposed to be
> bug fixes to a x.y.x kernel. In fact, the 2.0.x, 2.2.x, and 2.4.x kernels
> (all still supported) don't have this. The sensible idea is to look at
> the change log, and decide if you need the new kernel/package/what-ever. Is
> it a security update? Bug fix? Some new feature you can't live without? Or is
> it something else you don't need? Only you can tell for sure.
>
> Old guy

Thanks, Moe, for that lengthy and informative post.

I've studied the Makefile but it is rather large (716 lines) and there
are 63 mentions of the word 'install' (only 6 of cp and none of mv).

They are all rather obtuse, e.g.
install-pkgconfigDATA: $(pkgconfig_DATA)
and many point off towards other files like gamin.pc. It's a little much
to unravel, I think - or maybe I'm overwhelmed.

--
Toby.
Add the word afiduluminag to the subject
field to circumvent my email filters.
Ignore any mail delivery error.

[Moe Trin]

On Tue, 10 Jan 2006, in the Usenet newsgroup alt.os.linux.ubuntu, in article
<slrnds76uo.9dd.google@ID-171443.user.uni-berlin.de>, Toby Newman wrote:

>Thanks, Moe, for that lengthy and informative post.

Yeah, I do tend to get wordy - but I hope it provides the answers you need.

>I've studied the Makefile but it is rather large (716 lines) and there
>are 63 mentions of the word 'install' (only 6 of cp and none of mv).

The best way to learn about this is to start small, and 716 lines is not
small. The word 'install' will be used two or three ways.

install: This defines a 'target', or task to be run. You may
notice other targets, like 'clean:' 'distclean:', or similar. /bin/make
would run commands in that section when you 'make install', 'make clean'
or what-ever.

[compton ~]$ whatis install
install (1) - copy files and set their attributes
[compton ~]$

There is a command named 'install'. This is _like_ 'cp' except that it
can also create needed directories, set ownerships and permissions, and
so on all in one command (rather than using mkdir, chmod, chown and cp).

Finally, the word could be part of a comment - that _should_ (but may
not always) be obvious in context.

>They are all rather obtuse, e.g.
>install-pkgconfigDATA: $(pkgconfig_DATA)

Yeah, to understand some of this, it helps to have a large bucket of coffee
(tea in your case), the man pages for the shell and a bunch of commands, and
maybe the 'abs-guide' ([url]http://tldp.org/LDP/abs/html/index.html[/url]) and lots and
lots of patience. Make is a way of automating a compile and install, and it
usually is doing nothing more than running a {small|large} number of commands
normally run from a command prompt. Think of it as a shell script or a DOS
batch file, or similar. This is why I recommend starting with a very simple
Makefile, and getting the toes wet, before jumping in while hugging an
anchor from HMS Enormous or similar. You also want to watch very carefully
- I'm pretty sure you have at least one, maybe three typos in the line above
($(pkgconfig_DATA) probably should be ${pkgconfig_DATA} rather than using
parentheses, which would make it a shell variable which is defined further up
the page, rather than an index in C).

>and many point off towards other files like gamin.pc. It's a little much
>to unravel, I think - or maybe I'm overwhelmed.

Yup - that's what we're talking about when we suggest "start small".

Old guy