Problems install pyopencl to get nengo-ocl to work

Backends
1

Hi,

I have been following https://github.com/nengo/nengo-ocl to get nengo-ocl to work for me. Currently I’m stuck on step 3 of https://wiki.tiker.net/PyOpenCL/Installation/Linux. When I type “sudo make install” I get the below error. I’m not sure what to do. I’m working on Ubuntu 16.04. I think I correctly updated to Python 3.5. To configure I typed “python3 configure.py” which had no errors.

Any help would be greatly appreciated! Thank you!

Kate

mendat@hephaestus : ~/Downloads/pyopencl-2018.1.1 $ python3 configure.py --cl-inc-dir=/opencl-install-location/include --cl-lib-dir=/opencl-install-location/lib
mendat@hephaestus:~/Downloads/pyopencl-2018.1.1$ sudo make install
ctags -R src || true
ctags: skipping src: it is not a regular file.
/usr/bin/python3 setup.py install
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
compilation terminated.
---------------------------------------------------------------------------
Sorry, your build failed. Try rerunning configure.py with different options.
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.5/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "/usr/lib/python3.5/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/usr/lib/python3.5/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/usr/lib/python3.5/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 161, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 147, in call_command
    self.run_command(cmdname)
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 23, in run
    self.build()
  File "/usr/lib/python3.5/distutils/command/install_lib.py", line 109, in build
    self.run_command('build_ext')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 49, in run
    _build_ext.run(self)
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 338, in run
    self.build_extensions()
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 447, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 472, in _build_extensions_serial
    self.build_extension(ext)
  File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 174, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 532, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.5/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/lib/python3.5/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 243, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 273, in run
    return func()
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 242, in runner
    _execfile(setup_script, ns)
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-b7yzpkmj/cffi-1.11.5/setup.py", line 240, in <module>
    "pytools>=2017.6",
  File "/usr/lib/python3.5/distutils/core.py", line 163, in setup
    raise SystemExit("error: " + str(msg))
SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1087, in run_setup
    run_setup(setup_script, args)
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 246, in run_setup
    raise
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 166, in save_modules
    saved_exc.resume()
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 141, in resume
    six.reraise(type, exc, self._tb)
  File "/usr/lib/python3/dist-packages/pkg_resources/_vendor/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 243, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 273, in run
    return func()
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 242, in runner
    _execfile(setup_script, ns)
  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-b7yzpkmj/cffi-1.11.5/setup.py", line 240, in <module>
    "pytools>=2017.6",
  File "/usr/lib/python3.5/distutils/core.py", line 163, in setup
    raise SystemExit("error: " + str(msg))
SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 265, in <module>
    main()
  File "setup.py", line 261, in main
    zip_safe=False)
  File "/home/mendat/Downloads/pyopencl-2018.1.1/aksetup_helper.py", line 21, in setup
    setup(*args, **kwargs)
  File "/usr/lib/python3.5/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 269, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 313, in fetch_build_eggs
    replace_conflicting=True,
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 826, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1092, in best_match
    return self.obtain(req, installer)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1104, in obtain
    return installer(requirement)
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 380, in fetch_build_egg
    return cmd.easy_install(req)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 663, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 693, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 873, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1101, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1089, in run_setup
    raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Makefile:12: recipe for target 'install' failed
make: *** [install] Error 1
2

Could you try running:

sudo apt-get install libffi-dev python-dev

This suggestion comes from the first reply in https://github.com/Kozea/cairocffi/issues/14 after Googling the No package 'libffi' found message.

You might also need:

sudo apt-get install python3-cffi

since it seems to be using Python3.

3

That totally helped! Thank you so much! Now I can successfully make install.

After doing that though, I then went to try running:

py.test nengo_ocl/tests --pyargs nengo -v

and get the below error. I already “sudo pip install pyopencl” and did a developer install of nengo-ocl. Did I miss a step?

**mendat@hephaestus** : **~/Kate/git/nengo-ocl** $ py.test nengo_ocl/tests --pyargs nengo -v

Traceback (most recent call last):

File "/home/mendat/.local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 377, in _getconftestmodules

return self._path2confmods[path]

KeyError: local('/home/mendat/Kate/git/nengo-ocl/nengo_ocl/tests')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/home/mendat/.local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 408, in _importconftest

return self._conftestpath2mod[conftestpath]

KeyError: local('/home/mendat/Kate/git/nengo-ocl/nengo_ocl/tests/conftest.py')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/home/mendat/.local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 414, in _importconftest

mod = conftestpath.pyimport()

File "/home/mendat/.local/lib/python3.6/site-packages/py/_path/local.py", line 668, in pyimport

__import__(modname)

File "/home/mendat/Kate/git/nengo-ocl/nengo_ocl/__init__.py", line 5, in &lt;module&gt;

import pyopencl as cl

File "/usr/lib/python3/dist-packages/pyopencl/__init__.py", line 30, in &lt;module&gt;

import pyopencl._cl as _cl

ModuleNotFoundError: No module named 'pyopencl._cl'

ERROR: could not load /home/mendat/Kate/git/nengo-ocl/nengo_ocl/tests/conftest.py
4

Odd. The only mentions of that error (ModuleNotFoundError: No module named 'pyopencl._cl') I can find are in archived mailing lists from before 2010…

One thing that looks suspicious to me is that the path starts off as running py.test in /home/mendat/.local/lib/python3.6/site-packages/_pytest/config/.

But then pyopencl is in a completely different path: /usr/lib/python3/dist-packages/pyopencl/.

Have you been using a virtual environment for some of these steps, or know if you have multiple different versions of Python3, by any chance?

Can you try running:

python3 -c "import pyopencl; print(pyopencl.__file__)"

I expect the above command to fail with the same error message above. Regardless, it would be useful to see if there are any differences in the paths.

5

It actually doesn’t fail. It returns:

python3 -c “import pyopencl; print(pyopencl.file)”

/usr/local/lib/python3.5/dist-packages/pyopencl-2018.1.1-py3.5-linux-x86_64.egg/pyopencl/init.py

6

That’s great, actually! This means Python was able to import pyopencl, and so you should be able to go ahead and start using it.

The problem is you have at least 2 different versions of Python installed. The 3.5 version located at /usr/local/lib/python3.5, that was able to import pyopencl, and the 3.6 version located at /home/mendat/.local/lib/python3.6/ which py.test decided to use.

So… you should already be able to import nengo_ocl from Python3.5 and build some models!

Though, the thing to watch out for is that most of your packages could be on Python3.6. If you get any more missing imports, you will need to install them to the correct location via (sudo) python -m pip install package-name assuming which python points to the appropriate version of python (which currently it does).

Still, given the state of things, I expect weird package-related problems of this sort to resurface. I’d recommend using a “virtual environment” to configure the Python version and packages all together under a given name. If you ever have versions that are incompatible with each other (or need a different Python version), you can create new environments and switch between them. For these purposes, I’ve found conda to be easy to use (it’ll be the same thing you were doing in Telluride for Loihi)! There’s also this way of creating virtual environments which might not be as user-friendly.

Let us know how this goes!

7

Thanks! I created an environment and pip installed all the packages I need and was able to successfully run a model using nengo-ocl. Thanks SO much for all of your help! It runs SO much faster on the GPU.

1 Like