Message boards : Number crunching : crunching 2 Python tasks concurrently
Author | Message |
---|---|
I remember when way back there were these "short runs" ACEMD tasks, some people, including myself, made some settings in a config_app.xml with the effect that 2 such tasks were processed simultaneously with the same GPU. | |
ID: 59199 | Rating: 0 | rate: / Reply Quote | |
I ran 3x concurrently on my RTX 3060. was doing each task in about 13.5hrs (effectively 4.5hrs per task) for ones that run full-length. but now that I've added a second GPU, and 3 more tasks, the CPU can't keep up anymore. will update to a 16/32 part soon to see if it helps. | |
ID: 59200 | Rating: 0 | rate: / Reply Quote | |
thanks, Ian&Steve C., for the thorough explanation. | |
ID: 59201 | Rating: 0 | rate: / Reply Quote | |
You need something like this: <app_config> <app> <name>PythonGPU</name> <max_concurrent>1</max_concurrent> <gpu_versions> <gpu_usage>0.5</gpu_usage> <cpu_usage>2.0</cpu_usage> </gpu_versions> </app> </app_config> Put it into a file you call app_config.xml, put that into the GPUGrid project folder and re-read config files from the Boinc client, or just quit Boinc and start it again. As I'm not at home atm I'm not able to have a look at the proper name for the Python GPU tasks. It's something along PythonGPU, but I'm not certain it's this exact spelling. Boinc will tell you if it's not right, so you can change it afterwards. I use the configuration with two CPU-threads per work-unit on one of my Ryzens which does other things on the rest of the CPU. Not sure if it would work as well with 0.45 or something like that, so the PythonGPU task would just take whatever CPU-cycles it can get from the sixteen threads the machine has altogether. I run a MilkyWay task beside two GPUGrid Python tasks as the latter won't really satisfy the 1080. ____________ - - - - - - - - - - Greetings, Jens | |
ID: 59202 | Rating: 0 | rate: / Reply Quote | |
You need something like this: How do I run an Einstein@Home / Milkyway GPU task alongside one GPUGrid Python task? GPUGrid's app_config seems to decide only how many simultaneous GPUGrid tasks run. With GPUGrid's Python tasks, I only get intermittent GPU usage, with constant 40ish% CPU usage - with a Core i7-12700K and RTX 3080 FE. GPU-Z screenshot attached. I don't think I have enough VRAM to comfortably run 2 GPUGrid Python tasks - with a single task I get up to 4947MB VRAM usage, and the GPU only has 10240MB total. But Milkyway / Einstein tasks only use about 1GB, so I could run one GPUGrid task alongside one Einstein task; just don't know how to... ____________ | |
ID: 59296 | Rating: 0 | rate: / Reply Quote | |
Set all your projects to 0.5 gpu_usage and then restrict GPUGrid to only one max_concurrent PythonGPU task. | |
ID: 59298 | Rating: 0 | rate: / Reply Quote | |
You'll need an app_config.xml in your respective Milkyway or Einstein directories. <app_config> <app> <name>Milkyway</name> <max_concurrent>1</max_concurrent> <gpu_versions> <gpu_usage>0.5</gpu_usage> <cpu_usage>.04</cpu_usage> </gpu_versions> </app> </app_config> For Einstein there's a lot of application names. Use this app_config.xml with 'Milkyway' in the Einstein directory as well, and Einstein will tell you it's wrong and its actual appliction names. Einstein uses 1.0 for cpu_usage if on Nvidia or AMD GPUs. ____________ - - - - - - - - - - Greetings, Jens | |
ID: 59299 | Rating: 0 | rate: / Reply Quote | |
The biggest lie for the new python tasks is "0.9 CPU". | |
ID: 59588 | Rating: 0 | rate: / Reply Quote | |
I errored out on three work units (27347453, 27347587, and 27347506) while trying to run more than one GPU task on my single GPU. I don't know if they failed because of my app_config.xml file or not, but I did not error out on a single GPU task, 27346636. I was also running Rosetta@home concurrently but have suspended R@h while running GPUGrid. | |
ID: 59595 | Rating: 0 | rate: / Reply Quote | |
The tasks failed because you haven't properly set a large enough paging file for your host in Windows. | |
ID: 59596 | Rating: 0 | rate: / Reply Quote | |
If you increase your paging file size, you should have no issues successfully completing the work units. Thank you, Keith, for your patience and help. I should have looked at the stderr output. My NMVe C: drive is only 250GB, so I put a fixed 228000MB paging file in another SSD. (Now I need more RAM to run three or more GPU tasks simultaneously.) | |
ID: 59597 | Rating: 0 | rate: / Reply Quote | |
Now I need more RAM to run three or more GPU tasks simultaneously. how powerful is your CPU ? | |
ID: 59600 | Rating: 0 | rate: / Reply Quote | |
how powerful is your CPU ? Intel i9-10850K @4.9GHz, ten cores and hyperthreaded. It could be a limitation because it is utilized at 100% when running two tasks on the Nvidia Titan X (single GPU). | |
ID: 59601 | Rating: 0 | rate: / Reply Quote | |
how powerful is your CPU ? well, the CPU is definitely powerful enough to run 2 Pythons concurrently. And basically more than 2 Pythons cannot be downloaded anyway, unless one "tunes" the "coproc-Info.xml" in the BOINC folder. | |
ID: 59602 | Rating: 0 | rate: / Reply Quote | |
Message boards : Number crunching : crunching 2 Python tasks concurrently