Error running Pollination commands after opening grasshopper

[milog (Milo Gillot)]

I seem to be having issues running Pollination commands specifically after opening Grasshopper.

Before opening grasshopper:

After opening grasshopper:

PO.V1.43.8.0 (Mar 06, 2024)
System.ArgumentException: Command:
C:\Program Files/ladybug_tools\python\python -m honeybee validate model "C:\Users\MiloGillot\AppData\Local\Temp\240308 TM59 Template.3dm.hbjson"  --json
Python path configuration:
  PYTHONHOME = '\\?\C:\Users\MiloGillot\.rhinocode\py39-rh8'
  PYTHONPATH = (not set)
  program name = 'C:\Program Files/ladybug_tools\python\python'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = 'C:\\Program Files\\ladybug_tools\\python\\python.exe'
  sys.base_prefix = '\\\\?\\C:\\Users\\MiloGillot\\.rhinocode\\py39-rh8'
  sys.base_exec_prefix = '\\\\?\\C:\\Users\\MiloGillot\\.rhinocode\\py39-rh8'
  sys.platlibdir = 'lib'
  sys.executable = 'C:\\Program Files\\ladybug_tools\\python\\python.exe'
  sys.prefix = '\\\\?\\C:\\Users\\MiloGillot\\.rhinocode\\py39-rh8'
  sys.exec_prefix = '\\\\?\\C:\\Users\\MiloGillot\\.rhinocode\\py39-rh8'
  sys.path = [
    'C:\\Program Files\\ladybug_tools\\python\\python310.zip',
    '\\\\?\\C:\\Users\\MiloGillot\\.rhinocode\\py39-rh8\\DLLs',
    '\\\\?\\C:\\Users\\MiloGillot\\.rhinocode\\py39-rh8\\lib',
    'C:\\Program Files\\ladybug_tools\\python',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ImportError: bad magic number in 'encodings': b'a\r\r\n'

Current thread 0x0000337c (most recent call first):
  <no Python frame>
   at Core.Utility.ExePythonCommand(String argument, String& results)
   at Core.Utility.ValidateModel(String hbJsonPath, Boolean roomOverlapsOnly, ValidationReport& report)
   at Pollination.RH.RhinoCommands.PO_ValidateModel.RunCommand(RhinoDoc doc, RunMode mode)

Any idea what might be causing this issue, seems as though it might be related to my environment paths. This is what I currently have configured:

[mostapha]

Hi @milog,

Thank you for the update. I can recreate the issue on my end in Rhino 8 but not the other versions of Rhino. I suspect it has to do with how Rhino 8 tries to load the DLLs from a different version of Python.

@chriswmackey and @eirannejad should be able to provide better insight. Meanwhile, I’m updating my installation from RS3 to RS5 to see if that resolves the issue.

[mostapha]

Now I’m more confident that the issue is because of how Rhino 8 loads the Python dependencies after dropping a Python 3 Script component on the canvas.

I was able to run the commands with no issues before dropping the Python 3 component that triggered the installation of other dependencies.

@milog, we will need some help from McNeel and @eirannejad to be able to resolve this issue. Meanwhile, you have two options:

1. Use Rhino 7.
2. Use Rhino 8 but don’t use the Python 3 Script so it doesn’t install Python 3.9 dependencies that get in the way of Pollination.

[chriswmackey (Chris)]

Thanks @milog and @mostapha ,

I think this may actually be an issue that @mingbo might be able to address in the Rhino plugin without having to wait for McNeel to make a change. I sense that the root of the problem is that @eirannejad had to set the PYTHONHOME variable in Rhino 8 to something that references his python environment (in the .rhinocode folder). You can see at the start of the error message that it’s telling you:

PYTHONHOME = '\\?\C:\Users\MiloGillot\.rhinocode\py39-rh8'

Setting this PYTHONHOME variable to something that we are not using in LBT/Pollination messes up all of the commands that we make from our software to our own copy of Python 3.10 that we ship with Pollination and Ladybug Tools. When I talked to @eirannejad about this at the AEC Hackathon last year, he made it seem that resetting the PYTHONHOME back to nothing (as it used to be in Rhino 7) would break many of the ScriptEditor capabilities that he added. So I agreed to work around it by just setting the PYTHONHOME to an empty string before making any command line calls to the version of Python we ship with Pollination/Ladybug Tools. You can see an example of one of these changes here:

fix(subprocess): Overwrite PYTHONHOME variable to it works on Rhino 8

This has gotten everything in the Ladybug Tools plugin to work correctly in Rhino 8.

I would presume that, if @mingbo is able to do something similar before he calls the honeybee validate model command from Pollination Rhino, we can avoid this error that the Rhino plugin is throwing.

@mingbo , is any of what I am saying making sense here? If not, I may be able to help if you point me to the part of the Rhino plugin where you call the honeybee validate model command. We basically just need to call this command with a custom environment that does not have the PYTHONHOME variable set.

[chriswmackey (Chris)]

FYI, @milog . If you want to use Option 2. that @mostapha specified to work around the issue for now, the best way to ensure that you don’t get this error message in Rhino 8 is to just delete this folder:

C:\Users\[USERNAME]\.rhinocode

If you do that, you can have both Pollination Rhino and Grasshopper open at the same time without having anything broken in either Pollination Rhino or Ladybug Tools. But the error will come back as soon as you launch the Rhino 8 ScriptEditor or drop a Python 3 component on your canvas.

Also, bear in mind that this solution is just temporary. Hopefully, once we get @mingbo to call the commands without the Rhino 8 PYTHONHOME variable, you won’t have a need for this workaround anymore and you’ll be able to use the new Rhino 8 ScriptEditor and Python 3 components without issues.

[eirannejad (Ehsan Iran Nejad)]

Thanks a lot @chriswmackey to put all this in writing here. We can potentially makes some changes to make this easier but @milog I think it is just good practice to set/override the PYTHONHOME environment variable for your honeybee subprocess since it needs a different deployment of python.

Let me know if solves the problem and if now we can think of other ways

[eirannejad (Ehsan Iran Nejad)]

@chriswmackey I can potentially unset PYTHONHOME after I am loading the engine but I have to test this. Would you mind if I send you a 8.x build towards the end of the week to test?

For reference: RH-80931 PYTHONHOME causes conflicts with python subprocess

[chriswmackey (Chris)]

Thanks, @eirannejad ,

I am definitely happy to test any new builds you send me. I’ll likely be busy this Friday but I should be able to easily test anything before that.

I should also report that @mingbo just told me that he found a way to overwrite the PYTHONHOME variable from the methods he is using on the .NET side. So there should be a fix for this particular case that @milog brought up here very soon.

Long story short, there’s really no rush from our end, @eirannejad , though I think it would still be nice if Rhino did not set the PYTHONHOME variable for all processes by default. This would certainly make things easier for us in the future and could possibly help some other developers who are trying to work with a version of Python outside of the Rhino environment.

[milog (Milo Gillot)]

I’ll avoid deleting the .rhinocode folder as this is where we are developing our custom python packages for our own internal plug ins. Seems like a fix is on the horizon so I’ll hold off for now. I raised this as an issue as I was just concerned that the simulations might be affected in some way given these error messages.

[threegreen (green three)]

I also have this problem when I choose to run on local machine in pollinate. However, when I use the cloud or just the local component, they are just fine.

[mostapha]

Thank you everyone! This should be solved in the latest release (v1.43.12) that was released today.

Let us know if it didn’t solve the problem for you.