Stream has unknown format: data in Layout.add_pcell_variant

Comments

• Matthias

  November 2022

  @Ege_Bey The stream readers/writers are dynamically loaded. You have to supply them separately I assume. If pyInstaller does what I think it does, it will analyze the DLLs and package all dependencies. I cannot know about dynamically loaded libraries.

  The "db" core library will try to locate them next to itself in a folder called "db_plugins".

  So look for a folder called "db_plugins" and make sure the files there are installed too, at the same place as "klayout_db.dll" (the "db" core library).

  Matthias
• Mustafacc

  September 2023 edited September 2023

  Hi @Matthias @Ege_Bey

  I'm trying to build a similar application and ran into the same issue. I tried to build it with the db_plugins added with the command below

  pyinstaller --onefile --add-data "/home/user/miniconda3/envs/dreamsfab/lib/python3.11/site-packages/klayout/db_plugins:./db_plugins" klayout_app.py

  However, when the application executes layout.read, it returns error loading plugins errors:

  ERROR: Unable to load plugin: /tmp/_MEIWHRoRK/db_plugins/lib_cif_dbpi.cpython-311-x86_64-linux-gnu.so
  ERROR: Unable to load plugin: /tmp/_MEIWHRoRK/db_plugins/lib_dxf_dbpi.cpython-311-x86_64-linux-gnu.so
  ERROR: Unable to load plugin: /tmp/_MEIWHRoRK/db_plugins/lib_gds2_dbpi.cpython-311-x86_64-linux-gnu.so
  ERROR: Unable to load plugin: /tmp/_MEIWHRoRK/db_plugins/lib_lefdef_dbpi.cpython-311-x86_64-linux-gnu.so
  ERROR: Unable to load plugin: /tmp/_MEIWHRoRK/db_plugins/lib_magic_dbpi.cpython-311-x86_64-linux-gnu.so
  ERROR: Unable to load plugin: /tmp/_MEIWHRoRK/db_plugins/lib_net_tracer_dbpi.cpython-311-x86_64-linux-gnu.so
  ERROR: Unable to load plugin: /tmp/_MEIWHRoRK/db_plugins/lib_oasis_dbpi.cpython-311-x86_64-linux-gnu.so
  ERROR: Unable to load plugin: /tmp/_MEIWHRoRK/db_plugins/lib_pcb_dbpi.cpython-311-x86_64-linux-gnu.so
  Traceback (most recent call last):
    File "klayout_app_gui.py", line 146, in execute_cell_fix
    File "klayout_app/__init__.py", line 75, in replace_from_file
    File "klayout_app/__init__.py", line 39, in layout
  RuntimeError: Stream has unknown format: /home/user/Downloads/example.gds in Layout.read
  Aborted (core dumped)
  

  any thoughts on why this is happening? I also tried compiling it with older python versions. The application is able to read the layout well before building it.

  Thanks!
• Matthias

  September 2023

  Hi @Mustafacc,

  I ssume some .so dependencies are not satisfied. You can run the application with LD_DEBUG=files to get more information about what is actually happening. In the log you should see the attempts the loader performs and why it finally fails.

  I suspect that the db_plugins have been built with RPATH baked in - if you move them to /tmp, they will not find the dependency libraries.

  In addition, the "Aborted" as the end is weird, but without further information I can't say why this happens.

  Matthias
• Ege_Bey

  February 5

  I will attach this code in case anyone encounters the same error. Just like Matthias mentioned it is about db_plugins folder.
  The issue is pyinstaller is not adding db_plugins folder into klayout folder as dependency which is indicated with red marked in the screenshot.

  This screenshot is from _internal/klayout folder. (_internal is the folder created after compilation of your app. It appears if you use --onedir option in pyinstaller. If you don't want this folder and you want just a single executable use --onefile option.)

  

  For compilation I use the following script which automatically includes db_plugins folder into klayout folder as dependency in the compiled application. If you run this script or execute the same pyinstaller_command you should no longer experience this issue.

  import os
  import subprocess
  import klayout
  
  ##TO RUN THIS SCRIPT MAKE SURE THAT YOUR CONDA ENVIRONMENT IS ACTIVATED AND YOU INSTALLED KLAYOUT LIBRARY AND PYINSTALLER INTO YOUR CONDA ENVIRONMENT
  
  
  # Get the path to KLayout inside the Conda environment
  klayout_path = os.path.dirname(klayout.__file__)
  db_plugins_path = os.path.join(klayout_path, "db_plugins")
  
  exe_name = "my_application"
  
  
  add_data_option = f'"{db_plugins_path};klayout\db_plugins"'
  
  # Construct the PyInstaller command
  pyinstaller_command = [
      "pyinstaller",
      "--onedir",           # To create a directory instead of a single EXE use --onedir instead of --onefile
      "--hidden-import=klayout.pya",  # Ensure klayout.pya is included
      "--add-data", add_data_option,  # Include db_plugins dynamically
      "--name", exe_name,  # Output EXE name
      "main.py"  # Your main Python script
  ]
  
  # Run the command
  print("Running PyInstaller with dynamic db_plugins path...")
  subprocess.run(" ".join(pyinstaller_command), shell=True)
  
  print("\nCompilation completed. Check the 'dist/' directory.")