Oleksandr Tymoshenko | 1 Dec 05:06 2011

Ports cross-compilation

I've been tinkering with ports cross-compilation for a couple of days 
and decided to summarize this experience. It might start some
discussion, or, with any luck, some action.

I had embedded platforms on my mind and for this use-case we do not
need all the ports to be cross-compilable. What we need is set of tools
to make cross-compilation possible and with these tols porters can start
adopting existing ports and explicitly marking them as cross-compilable.

With this objective I started to add hacks to ports/Mk and tried to
create several generic applications. Here is what I have discovered
so  far:

- xdev-generated compilers can be used as toolchains for ports
   cross-compilation. Some adaptation are required due to different
   platform naming in GNU configure (e.g. mips64eb vs mips64)

- Ports lack "buildroot" notion. There is no distinction between TARGET
   directory and PREFIX.  Target directory is always assumed to be root
   directory. DESTDIR knob tries to fix this but it's more of a shortcut
   than proper solution - it just places installation process in chrooted
   environment leaving it agnostic of real target directory

- Dependencies should be split in two groups: host dependencies (build
     tools) and target dependencies (libraries)

- Package builder works only on installed port. It allows some ports to
   make no distinction between two steps: (a) generate files tree for
   packaging and (b) generate files during installation. e.g.
   pre-compiling function files for shells/zsh: port tries to do it as
(Continue reading)

Stefan Bethke | 1 Dec 08:44 2011
Picon

Re: Ports cross-compilation

Am 01.12.2011 um 05:06 schrieb Oleksandr Tymoshenko:

> I've been tinkering with ports cross-compilation for a couple of days and decided to summarize this
experience. It might start some discussion, or, with any luck, some action.

Excellent points!

> - Package builder works only on installed port.

Have you looked at pkgng yet?  The wiki page says it can create a package from a separate directory tree.

> - Makefile for cross-compilable port should be split into three parts:
>  common, native, cross. It's not clear who should maintain cross part
>  though.

From many previous discussions, people are reluctant to add files to all ports because of the filesystem
and VCS bloat that causes.  Also, considering the number of ports there are in the tree, and how well
maintained many of the lesser ones are, any solution that requires no or very little changes to each port
would stand a much bigger chance of being implemented successfully.

Has anyone set up a ports build for mips yet, perhaps in an emulator?  It would be very interesting to see which
ports build at all on mips.

(I naively installed editors/joe from my TL-WR1043ND, which took about a day mainly because it pulls in
perl, and the perl build tools grow to around 100 MB memory size.  Most of the time was spent paging…)

Stefan

--

-- 
Stefan Bethke <stb@...>   Fon +49 151 14070811
(Continue reading)

Adrian Chadd | 1 Dec 08:49 2011
Picon

Re: Ports cross-compilation

The ports I need build fine on MIPS. I do it all the time on my
routerstation pro (680mhz mips24k, 128mb RAM.) Be careful if you
compile C++ code, the paging hurts. :)

Cross-compiling though, that'd be nice. :)

Adrian
Oleksandr Tymoshenko | 1 Dec 09:00 2011

Re: Ports cross-compilation

> 
>> - Package builder works only on installed port.
> 
> Have you looked at pkgng yet?  The wiki page says it can create a package from a separate directory tree.
   No, not yet. By "Package builder" I meant package-building 
targets of ports Makefiles. pkg_create can work on directory 
tree + setof pre-generated files. It's just that at the moment 
we use "pkg_create -b" to create package archive.

>> - Makefile for cross-compilable port should be split into three parts:
>> common, native, cross. It's not clear who should maintain cross part
>> though.
> 
> From many previous discussions, people are reluctant to add files to all ports because of the filesystem
and VCS bloat that causes.  Also, considering the number of ports there are in the tree, and how well
maintained many of the lesser ones are, any solution that requires no or very little changes to each port
would stand a much bigger chance of being implemented successfully.

As I told - getting all ports cross-compilable is impossible. 
We're talking about most-used in embedded environment ports. I'd say 
it's a couple of hundreds. So we need modify only these ports and only 
if it's really required. Simple ports like converters/base64 will not 
require modification at all. 
Adrian Chadd | 1 Dec 09:15 2011
Picon

Re: Ports cross-compilation

.. hm. thinking about it, why not have a port variable flag that marks
a port as "cross compiles" ?

Then we could (in theory) do a cross-compile test run based on which
ports in the ports tree have this variable set?

Adrian
Lev Serebryakov | 1 Dec 10:51 2011
Picon

Re: Ports cross-compilation

Hello, Oleksandr.
You wrote 1 декабря 2011 г., 8:06:31:

> - Some ports rely on autoconf's ac_try_run test to get some information
>    about target platform. It fails for cross compilation of course.
>    Few fail graciously by asking developer to provide this information
>    as #define, some just stick with default value, some bluntly crash.
>    This should be dealt with at per-port basis.
                                                 ...by upstream maintainers :)

--

-- 
// Black Lion AKA Lev Serebryakov <lev@...>

Warner Losh | 1 Dec 17:09 2011

Re: Ports cross-compilation


On Dec 1, 2011, at 1:00 AM, Oleksandr Tymoshenko wrote:

>> 
>>> - Package builder works only on installed port.
>> 
>> Have you looked at pkgng yet?  The wiki page says it can create a package from a separate directory tree.
>   No, not yet. By "Package builder" I meant package-building 
> targets of ports Makefiles. pkg_create can work on directory 
> tree + setof pre-generated files. It's just that at the moment 
> we use "pkg_create -b" to create package archive.
> 
> 
>>> - Makefile for cross-compilable port should be split into three parts:
>>> common, native, cross. It's not clear who should maintain cross part
>>> though.
>> 
>> From many previous discussions, people are reluctant to add files to all ports because of the filesystem
and VCS bloat that causes.  Also, considering the number of ports there are in the tree, and how well
maintained many of the lesser ones are, any solution that requires no or very little changes to each port
would stand a much bigger chance of being implemented successfully.
> 
> As I told - getting all ports cross-compilable is impossible. 
> We're talking about most-used in embedded environment ports. I'd say 
> it's a couple of hundreds. So we need modify only these ports and only 
> if it's really required. Simple ports like converters/base64 will not 
> require modification at all.

When I experimented with this years ago, I found that only a small number of ports needed extra files.  Many
worked because they were either (a) so trivial that they just needed a different compiler or (b) had
(Continue reading)

Warner Losh | 1 Dec 17:12 2011

Re: Ports cross-compilation


On Dec 1, 2011, at 1:15 AM, Adrian Chadd wrote:

> .. hm. thinking about it, why not have a port variable flag that marks
> a port as "cross compiles" ?
> 
> Then we could (in theory) do a cross-compile test run based on which
> ports in the ports tree have this variable set?

In the doodle I did years ago, I had a CROSS_BUILD_FLAVOR = {trivial, gnuconf, custom} and had that drive
some of the infrastructure.  This didn't make it into the final hack we used at symmetricom, however.  Some
of that can be inferred from other variables, but we didn't bother to try to build stuff we didn't need and
that might not be working.

Warner

Adrian Chadd | 2 Dec 15:39 2011
Picon

Re: TL-WR1043: switch

.. erk.

Well, what shall we do? Create a different driver? Or just enable a
"quirk" or configuration paramater, which allows the device to be
not-quite-i2c?

Adrian
Aleksandr Rybalko | 2 Dec 15:45 2011
Picon

Re: TL-WR1043: switch

On Fri, 2 Dec 2011 22:39:32 +0800
Adrian Chadd <adrian@...> wrote:

>> .. erk.
>> 
>> Well, what shall we do? Create a different driver? Or just enable a
>> "quirk" or configuration paramater, which allows the device to be
>> not-quite-i2c?

It's too specific to Realtek, so from one point better to have
separate driver, but from another point it have i2c EEPROM on same bus.
So if we want to have access to both, we need quirk.

>> 
>> 
>> Adrian

WBW
--

-- 
Alexandr Rybalko <ray@...> 
aka Alex RAY <ray@...>

Gmane