Python, distutils and IBM XL compilers

[sawyl]

This may have been fixed in later versions of Python — I've been using 2.6.6 — but I've noticed that the standard distutils don't seem to work properly when (a) the interpreter has been built using the the XL compilers; and (b) when the extension being built needs to be linked with the C++ compiler, e.g. in SciPy. When both these conditions are met, distutils generates an invalid command link that features both the C++ compiler as the linker (correct) and C compiler as the second argument (horribly incorrect), as this example from an existing problem ticket shows:

error: Command "xlC xlC -qlanglvl=extc89 ..." failed with exit status 252

Not being willing (or able) to build each component by hand, I came up with a rough and ready fix — the if block that follows the platform test for 'aix5' — for distutils.unixccompiler.link() that seems to work around the problem:

    def link(self, target_desc, objects,
             output_filename, output_dir=None, libraries=None,
             library_dirs=None, runtime_library_dirs=None,
             export_symbols=None, debug=0, extra_preargs=None,
             extra_postargs=None, build_temp=None, target_lang=None):
        objects, output_dir = self._fix_object_args(objects, output_dir)
        libraries, library_dirs, runtime_library_dirs = \
            self._fix_lib_args(libraries, library_dirs, runtime_library_dirs)

        lib_opts = gen_lib_options(self, library_dirs, runtime_library_dirs,
                                   libraries)
        if type(output_dir) not in (StringType, NoneType):
            raise TypeError, "'output_dir' must be a string or None"
        if output_dir is not None:
            output_filename = os.path.join(output_dir, output_filename)

        if self._need_link(objects, output_filename):
            ld_args = (objects + self.objects +
                       lib_opts + ['-o', output_filename])
            if debug:
                ld_args[:0] = ['-g']
            if extra_preargs:
                ld_args[:0] = extra_preargs
            if extra_postargs:
                ld_args.extend(extra_postargs)
            self.mkpath(os.path.dirname(output_filename))
            try:
                if target_desc == CCompiler.EXECUTABLE:
                    linker = self.linker_exe[:]
                else:
                    linker = self.linker_so[:]
                if target_lang == "c++" and self.compiler_cxx:
                    # skip over environment variable settings if /usr/bin/env
                    # is used to set up the linker's environment.
                    # This is needed on OSX. Note: this assumes that the
                    # normal and C++ compiler have the same environment
                    # settings.
                    i = 0
                    if os.path.basename(linker[0]) == "env":
                        i = 1
                        while '=' in linker[i]:
                            i = i + 1

                    linker[i] = self.compiler_cxx[i]

                    if sys.platform == 'aix5':
                        del(linker[1])
                        ld_args.append('-bnoentry')
                        ld_args.append('-G')

                if sys.platform == 'darwin':
                    linker = _darwin_compiler_fixup(linker, ld_args)

                self.spawn(linker + ld_args)
            except DistutilsExecError, msg:
                raise LinkError, msg
        else:
            log.debug("skipping %s (up-to-date)", output_filename)

It's a long way from perfect — the test shouldn't really be for AIX but something like self.compiler_cxx.startswith("xl") and the code should search through the list, rather than simply removing element 1 — but it's a reasonable first cut at a solution to a deeply annoying problem.