Wanted to ask about the result files and how you recommend managing them.
I see that the simulation files (inputs/outputs) are downloaded to C:\Users\USER_NAME\AppData\Local\Temp\Pollination.
It means that, in theory, once they are in the local machine i don’t need to connect to the pollination site, right? If so, this is the right thing to do? I suppose so, but i’m asking because maybe you are planning on something else on this regard. If this is the expected workflow, maybe you can provide an input for the folder to download the results instead of the previous path.
Two minor additional comments:
The inputs folder contains only the JSON model. Can be convenient including also the wea and additional files that can be part of the simulation.
I see that in addition of the simulation folder (that includes inputs and outputs) another couple of folders are downloaded (at least for the case i simulated, radiation. One with the direct and another for the total). I don’t see they are used at all.
Currently, we only support the “cloud jobs” which means you will need to connect to the internet to check the job status and download job’s assets (inputs/outputs) even if you have outputs saved to local drive. But when downloading the files, the component will try to reuse the cached files when a asset is already downloaded (you can right click the component to disable it).
But this will be a different story when we enable running jobs on the local machine. In that case, all components should work fine without connecting to the pollination site. (except running a job with a private recipe locally)
The inputs folder contains only the JSON model. Can be convenient including also the wea and additional files that can be part of the simulation.
You can download the wea and other input files with “Check Run Inputs” component as well. The Json file is just one of the inputs of the recipe.
I see that in addition of the simulation folder (that includes inputs and outputs) another couple of folders are downloaded (at least for the case i simulated, radiation. One with the direct and another for the total). I don’t see they are used at all.
The component doesn’t get to pick what content a output asset contains, it is defined in the recipe and up to the author of the recipe.
In your specific case, for radiation study, may be @Mostapha can help with this question.
Hi @Mingbo,
Thanks for your prompt answer. Some questions back:
But once the job is done and i checked status and downloaded results, if i move them to another location on the local machine, next time i want to see the same results i don’t need to check anything but just connect the result files to the proper LBT component to see the results, right?
This is not happening with the radiation simulation i’m testing right now. Each time i check status it downloads 2 separate folders withe Total and Direct results. It is true that the files used for the visualization are taken from the cached/existing folder. So I don’t understand why those 2 folders are good for. @mostapha will make things clear here.
My questions here are intended only to understand the workflow …
That’s correct. Even from the same location you can reuse the results. This is intentional and part of the design principles. You have the freedom to take the data and visualize them in any other platform including Ladybug Tools Grasshopper plugin.
See “your data is your data” section in this post:
This is how the recipe is written/designed to separate the results for total irradiance from direct irradiance. If you are interested to get the total (direct + indirect) radiation then you should look to the results inside total folder. For cases that you’re interested in direct sunlight (no indirect contribution) then you should check the results in direct-radiation folder.
Our recipe viewer still need some work to be really helpful but you can see that in the process of Radiation study the study is calculated for direct-sunlight and indirect sunlight separately to ensure accurate results for direct sun.
Maybe i didn’t explain myself good enough. I understand the 2 folders. This is fine. I was asking why those 2 folders are downloaded each time i reconnect the job results. They are already in the computer. I tested reconnecting the job results 3-4 times and both folders were downloaded every time. See for instance this image.
The directory named 3483ac0b includes the inputs/outputs. All the other are pairs of outputs (direct/total) for the same simulation. I wonder here if this can be spared. For sure they are not used in the GH file.
Now I see! Sorry if I was confused. This looks like a bug to me. The folder name should match the run ID. That will ensure the results will be downloaded only once. @Mingbo, do you know what’s going on here?
“3483ac0b” folder is the run assets folder in which the component checks for cached files.
All other folders in your screenshot are temporary folders that component unzips downloaded assets to (each output asset are zipped).
So here is the workflow of how component downloads the run assets:
Check the job/run status from the server
Check SaveAsFolder/RUNID (this case: TEMP/Pollination/3483ac0b) to see if any of assets are downloaded.
Download the asset zip file, and unzip it to TEMP/Pollination, and copy them to TEMP/Pollination/3483ac0b/Outputs/AssetName.
Loads results to GH.
Here is a demo to show that the component won’t redownload the assets twice if they are already exist in the TEMP/Pollination/3483ac0b/Outputs/. Those temporary folder for unzipping files could be safely deleted, and all files in TEMP file will be cleaned by Windows.
Hi @Mingbo, thank you for the explanation. Are you saying that those temporary folders are created for other runs? If you create a new folder every time then there will be no use of cached results really.
Those temporary folders are only for unzipping zip files when there is no cached files found and it needs to download them from the server.
Once it finds the cached files under run id, it won’t re-download the results’ zip file, so there won’t be any new temporary folders.
I see. Thanks! @ayezioro, can you confirm that this is the behavior that you see on your side?
Also now that I see how Abraham thinks about this should we create the temp folder for downloading results in some other folder and keep the temp folder for pollination clean?
Still don’t get it. Sorry.
I deleted all contents from the Temp/pollination just to start fresh.
Then i pick a job in the Check Job Status component. This is what happens in the pollination dir:
I believe i’m doing what @Mingbo shows in the animation above.
Attached the file i’m using for this case. Maybe you can find what i’m doing wrong. Radiation_12.4.2021.gh (109.4 KB)
As i said, no. Temporary dirs are duplicating each time.
There is something i need to update or this is happenning on backstage?
I started with clean folder, and it creates three folders (3483ac0b is the run folder, and the other two are for two output assets: direct and total results).
I repeated what you did: disable and enable the Job Status, I don’t see it creates another two more folders.
Thanks for checking @ayezioro! Most likely the code is looking for the wrong file and assumes it doesn’t exist which results in the additional download. In @Mingbo we believe! He should be able to help you with this issue.
Just reviving this old topic, I was wondering if this is still happening to other users. I dont recall that this was happening to me earlier. However, after the recent firm-wide Windows 11 upgrade, when I try to bring back cloud sim results, Pollination creates two result folders in two different locations simultaneously (with entirely different naming).
That’s the expected behavior. Pollination downloads the files into the %TEMP% folder first and then copies them to the target folder. The %TEMP% folder gets cleaned up by the system automatically.
He should be able to help you with this issue.