why are these CPU jobs for Linux only, and not for Windows, too?

Because they can make the app work under Linux but are not successful yet in creating a Windows app that works.

Because they can make the app work under Linux but are not successful yet in creating a Windows app that works.

Erich56 said:

further, an easy way to solve this would be to have the Linux app run in a Virtual Machine

I have been running CERN LHC@home Virtual Machines for more than ten years, and I have been rewarded with a CERN Polo Shirt. But yes, they do present some problems. Now your CPU tasks seem to run fine on my old SUN Workstation with SuSE Leap 42.3 Linux.
Tullio

That is what I have done on my Windows machines and it works fine.

The recent batch of CPU WUs seems to be done. Will there be more soon?

No issue at all. I'm glad the team communicates openly.

Thank you for the heads-up.

All of my Stefan CPU WUs are stuck at 10% and I aborted them after about 4 hours. This is the machine (16.04 LTS) that has never had any issues with pretty much any of the WUs.

Here is the Error page: https://www.gpugrid.net/results.php?hostid=424454&offset=0&show_names=0&state=5&appid=

Holy cow the website is SOO SLOW. I had to use a proxy in Sweden to just get anything to load. I can't even get tasks even though the site says there are plenty.

1 error recently:
http://gpugrid.net/result.php?resultid=18003621

Why is the project requesting 28 GB of disk space? I see a three files stuck downloading that are less than 10kb combined. Project folder and slot folders are nowhere close to 28GB.
GPUGRID 7/5/2018 10:18:01 PM Message from server: Quantum Chemistry, beta test needs 17080.43MB more disk space. You currently have 11529.80 MB available and it needs 28610.23 MB.

These are taking over 4.2-4.5 GB of memory? I thought this was much lower before? That PC has 128GB but Mint split the disk space so not a lot of disk space. But 28 GB is a bit much.

GPUGRID is taking 3.34 GB of disk space on my main Linux host, 3.90 on a Linux laptop. On the same laptop LHC@home is taking 5.75 GB.
Tullio

http://gpugrid.net/results.php?hostid=470907

I have a lot of errors :(

Must we update conda?
Please update conda by running

$ conda update -n base conda
Tullio

They are much larger molecules so they were crashing on the QM queue cause they ran out of scratch space. I have seen them use up to 18GB scratch space so at the moment I don't know yet how to run these on GPUGRID as it seems to be an issue with many users.

We are using QC beta to test large molcules and how much disk space they take. I think they can (temporarily of course) go up to 20 GB of space (!). I am not sure about RAM - they should be < 4 GB.

We'll need to think whether to split the app in large and small.

Regarding conda: it's an "internal" installation. So no, you can't update, and you shouldn't. That message is unfortunate.

Best

As far as I know virtualization is almost native speed these days, especially for computing.

We are using QC beta to test large molcules and how much disk space they take.

I have a degree in Theoretical physics obtained in 1967, but that was related to elementary particle physics. Then in the Nineties, while at Trieste Area Science Park as manager of a UNIX BULL Laboratory I attended a few lectures in the UN Center for Genetic Engineering and Biotechnology on the Density Functional Theory. Since retirement, I have run a few BOINC projects including one on Monte Carlo Method applied to Quantum Chemistry but it no longer exists. This is the first time I am running a project which uses Neural Networks.
Tullio

Well... So my Threadripper could need up to 160GB of space?! It has just 32GB...

We are talking about DISK space. Only a few WUs will be that big - unless we make a "big" queue.

T

I had to enlarge the root partition to accommodate the QC beta 3.31 since boinc from the Fedora distro by default installs in /var/lib/boinc and runs as a daemon under systemctl. After that, the WU's seemed to run fine but they sure ate up a lot of RAM. Both my 8 core's have 16 GB RAM and I was running them 2 concurrent with 4 cores each. I think only two errored out and the rest completed and validated. Guess I'll have to max my 8 core machines with 32 GB RAM to run the bigger molecules.
____________

Crunching since Feb 2003 (United Devices, Find-a-Drug)

Easier and more precise simply to set max_concurrent in app_config.xml

Worth it to perform the experiment, certainly. Possibly depends whether it respects the declared space needed (<rsc_disk_bound>), or the actual space used. If the latter, there might be a problem if the actual usage increases gradually during the run - BOINC might only check it when deciding whether to start a(nother) new task. Lots of fun to be had with those possibilities...

@Toni, Yes boinc does appear to respect the client disk settings as it lets one know if disk space is too low to run certain projects in the event log. I usually set a high arbitrary GB size but the client appears to react to the real amount available in the execution partition and uses the percentage limits to notify user when disk space is too low. I had to readjust the percent limits higher (in the client settings) a few days ago to run the 3.30 app on one of my machines. Probably due to the project directory getting too full. I hate to reset the project and loose WU's but I suppose I will have to eventually.

We are using QC beta to test large molcules and how much disk space they take.

Toni, if I may ask you, what molecule size are we (roughly) talking about? As you know, because of my son I have personal interest in HCF1 research, and I would like to get a feel how far science is still away from handling that large molecules. Thanks in advance and my apologies for coming up with my personal issues once in a while.

Toni, if I may ask you, what molecule size are we (roughly) talking about?

The molecules for QM are max 50 atoms or so. The size is however not very indicative. This is a specific "chemistry-oriented" type of calculations.

On an EDX online course on quantum computers which I followed recently there was a professor at Dartmouth University who uses a quantum computer to do quantum chemistry calculations.
Tullio

Thank you very much. Which kind of contribution will help you most in order to make progress on proteins (in the long run of course)? Because I am just considering whether to buy an additional GPU or CPU this autumn.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

Thank you... no problem. So we just keep on crunching on all sides and see where the road leads us to. :)
By the way, congratulations to all of you guys at Gpugrid on your great work.
____________
I would love to see HCF1 protein folding and interaction simulations to help my little boy... someday.

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid. I'm thinking of leaving it crunching these instead of Rosetta (the rest of my PCs all run Windows...). Hope it helps!
____________

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid. I'm thinking of leaving it crunching these instead of Rosetta (the rest of my PCs all run Windows...). Hope it helps!

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid. I'm thinking of leaving it crunching these instead of Rosetta (the rest of my PCs all run Windows...). Hope it helps!

Wow! That's a lot of compute!

48 QM WUs are 192 (CPU) threads. I need 4 computers to reach that.

48 QM WUs are 192 (CPU) threads. I need 4 computers to reach that.

Well i run QC on a 2core pc just for fun :-)

But now i get; No task sent
This computer has finished a daily quota of 8 task


That´s not fun!!

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid.

Most tasks benefit from HT but I only recall one doing better overall with HT off on my 2670v1s.

The new Threadrippers with 4 die but only 2 with direct memory access, meaning 2 die have to always make a hop to access memory, would make for some interesting testing.

I have a related question I cannot answer myself.

I am seeing very unsteady CPU and memory usage with QC. CPU% drops to 70-80%, with MEM% dropping as well, (however they are defined by ubuntu) after a few seconds, then both peak again.

1) Is this a function of the algorithm or a limitation of my system?

2) Can I limit the number of cores used by QC?

As it happens I've found an issue today where BOINC crunches 3 WCG and one 4 core QC task at the same time on a four core processor. CPU usage has been stuck at 7.x for an hour at least.

QC uses 4 cores by default and I haven't found an option to limited the number of cores. Also, QC seems to be hogging my CPU at most times, eventhough it has been added a week ago. This has led to a bottleneck with WCG tasks due soon.

2) Can I limit the number of cores used by QC?

As it happens I've found an issue today where BOINC crunches 3 WCG and one 4 core QC task at the same time on a four core processor. CPU usage has been stuck at 7.x for an hour at least.

QC uses 4 cores by default and I haven't found an option to limited the number of cores. Also, QC seems to be hogging my CPU at most times, eventhough it has been added a week ago. This has led to a bottleneck with WCG tasks due soon.

Use an "app_config.xml" file to limit the number of cores per work unit, and also the number of QC work units running if you wish.

Use an "app_config.xml" file to limit the number of cores per work unit, and also the number of QC work units running if you wish.

Thanks Jim1348, I just tried, but without improvement. Seems to be connected to the algorithm.

You must tell BOINC to reread configs to pick up the changes. Tasks already downloaded will still say 4c even. Only new ones will say 1c or 2c but all will run at your new setting.

Its a BOINC thing to sometimes squeeze in more tasks than cores. I've seen it happen on my 3570k when a single threaded task completes and a 4c tasks starts it will show more running for a but but it eventually corrects.

Looks like the CPU WU Queue is almost running dry

I downloaded 4 QC tasks on my Windows 10 PC and of course they failed. But why the server sends me QC tasks on a Windows PC?
Tullio

I downloaded 4 QC tasks on my Windows 10 PC and of course they failed. But why the server sends me QC tasks on a Windows PC?
Tullio

Just a notice to Stefan, only a few days left of CPU WUs in the queue.

My HP Linux laptop running SuSE Leap 15.0 after Leap 42.3 (any relationship to SLES 15.0 ?) has 752.37 GB available to BOINC. Instead my older SUN WS running SuSE Leap 42.3 has at most 30 GB available to BOINC 7.8.3 of a 1 TB disk.
Tullio

I have a QC task running on my Linux laptop. It is at 73% after 9;07;27 hours. But its slot is empty.
Tullio

Two QC tasks failed on my main Linux box which has a 30 GB limit to BOINC 7.8.3 with the same message DISK USAGE LIMIT EXCEEDED. GPU task running fine on its GTX 750 Ti at 61 C.
Tullio
Same error after 55 minutes also on the laptop despite the 750 GB available to BOINC. But where is that scratch file? I cannot see in either in /tmp or /var/lib/boinc/slots.
____________

OK I have a feeling we hit a file-size limit of BOINC and not of the drives. I'll chat it up with Toni and see what we can do.

For the moment I'll cancel them since they nearly all fail with the same disk usage error.

Completed and validated two QC task on my main Linux host. A GPU task is running on it at 1202 MHz clock,5400 MHz memory transfer, temperature 61 C on its GTX 750 Ti, driver 384.111.
Tullio
____________

Completed and validated two QC task on my main Linux host.

just get Linux installed.

Two more QC tasks completed, two ready to start. Thanks.
Tullio
____________

CPU usage reaches 197% on my old Opteron 1210 with 2 cores, 145% when a GPU task is also running. RAM is 8 GB.
Tullio

CPU usage reaches 197% on my old Opteron 1210 with 2 cores, 145% when a GPU task is also running. RAM is 8 GB.
Tullio

new WUs don't seem to work: they consume a lot of memory, throw computation errors or just rest at 10% progress forever.

It's running at 1.8 GHz and I have a 1220 Opteron in my drawers at 2.8 GHz. It's been running since January 2008.My electricity costs me 0.21 euro /kWh and I have 3 computers running 24/7, this Opteron, an AMD E-450 and a A10-6700 which should have 4 cores but Windows Task Manager says 2 cores and 4 logical processors. My total electricity expenditure is about 60 euro/month.
Tullio
I forgot to mention my ulefone smart phone with its arm64-v8a CPU running Android 7.1.1 on SETI@home and Einstein@home.

I have an Intel 8 core (16 thread) Xeon server that has a 146 GB disk drive (has 2 of them but one died). It also has 24GB RAM.
WUs are allowed to run with 8 cores.

I am getting the message that I need 28610.23 MB Disk Space, I currently have 9486.42 MB spare, so it needs another 19123.81 MB of Disk Space.

I leave 10GB that BOINC can't use, other programmes use 12.69 GB, BOINC is using 17.02 GB.

Of that 17.02 GB that BOINC is using, GPU Grid is using 8.29 GB, even when it is not running anything.

If I allow all my spare space to be used I would just have enough disk space for GPUGrid to run (maybe), however I don't intend to give all that space to BOINC so I can't download and run some of these work units.

If they are 6GB then there is no problem.

Why does GPUGrid need over 8 GB of disk space just to hold the project files?
(I have another computer that is showing the same amount of used disk space so this is normal amount used by the project but Why?

(My other computer has a much larger Disk so is not having the same issues).

Conan

First SELE task done by my Old Faithful Opteron 1210 running SuSE Linux Leap 42.3.
Tullio

I have a funny SELE task on my Linux laptop. It is stuck at 10% after 14 hours 38 min, but the remaining estimated time is rising to more than 5 days. All seems normal by the "top" command and it has lots of disk space.
Tullio
____________

new WUs don't seem to work: they consume a lot of memory, throw computation errors or just rest at 10% progress forever.

On your failures I see "connection errors". Could be firewall filtering, or the like.

As said, those WUs do not work properly. I am away for another project and come back, if they are fixed.

OK, thanks Toni and Stefan for the information, that explains a lot.

I will run what I can.

Thanks again
Conan

In the slot of a running task there is an output directory which leads to a report of what the program is doing in physical terms. Maybe some explanation by the admins would be welcome.
Tullio
____________

I got plenty of disk space on my two Linux boxes because the slots directory is in my /home/user partition,which has more than 700 GB on my SuSE Linux Leap 42.3 and Leap 15.0 OS. What amazes me is that QC tasks are always stuck at 10% progress while GPU tasks show progress as increasing.
Tullio
____________

I got plenty of disk space on my two Linux boxes because the slots directory is in my /home/user partition,which has more than 700 GB on my SuSE Linux Leap 42.3 and Leap 15.0 OS. What amazes me is that QC tasks are always stuck at 10% progress while GPU tasks show progress as increasing.
Tullio

The 10% progress is explained as follows: updating (if necessary) the app is 10%, and usually happens immediately. The remaining 90% advances when molecules are calculated (e.g. 5 molecules = 90%/5 increments). However very big WUs have only one molecule, so no apparent progress until the end. (We have no finer grain progress).

In the old UNIX days a rule of thumb was that you needed a swap space twice the RAM, which was usually small.Now RAM is plenty. I got 22 GB RAM on the Windows 10 PC, and 8 GB RAM on each Linux box. GGPUGRID CPU tasks use some swap but most is not used.
tullio

I amped the swap to 300GB, but it only seems to be using RAM. Is this "scratch space" used in swap space or does the WU use the file directory for storage? I'm thinking it is the latter since the BOINC space usage goes up and down.

Thing is my install directory is only 120GB...

I also have this feeling that now I have 300GB of swap space for nothing lol. I am not a smart man.
____________

I see temporary files in the slots/0 directory They are named psi.25019.number
Tullio

Stefan, I see 4 plus one which says psi.30091.clean
Tullio

Im fixing an issue with SELE2 so I cancelled them and will send out SELE3 in a bit

I had to add WCG to this 48-thread beast because it isn't using all of the threads @ 100% when running GPUGRID only. I'd wager it's because of scratch space bottleneck (it's running a SSD tho, 200 MB/s according to hdparm)... ?
____________

No, the issue was with an old version of psi4 giving wrong results on large molecules when using the scratch space. This is fixed in the latest version now.

I am running 3.31 SELE6.
Tullio

Something is wrong. The BOINC Manager says it is running but python does not appear in the "top" console.

____________

I just had about 20 of these fly through before they corrected and started to run correctly.

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)</message>
<stderr_txt>
10:09:04 (14352): wrapper (7.7.26016): starting
10:09:04 (14352): wrapper (7.7.26016): starting
10:09:04 (14352): wrapper: running /usr/bin/flock (/home/zalster/Desktop/BOINC/projects/www.gpugrid.net/miniconda.lock -c "/bin/bash ./miniconda-installer.sh -b -u -p /home/zalster/Desktop/BOINC/projects/www.gpugrid.net/miniconda &&
/home/zalster/Desktop/BOINC/projects/www.gpugrid.net/miniconda/bin/conda install -m -y -n qmml2 --override-channels -c defaults -c gpugrid --file requirements.txt ")
Python 3.6.5 :: Anaconda, Inc.

PackagesNotFoundError: The following packages are not available from current channels:

- psi4==1.2.1

Current channels:

- https://repo.anaconda.com/pkgs/main/linux-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/linux-64
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/linux-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/pro/linux-64
- https://repo.anaconda.com/pkgs/pro/noarch
- https://conda.anaconda.org/gpugrid/linux-64
- https://conda.anaconda.org/gpugrid/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.


10:09:21 (14352): /usr/bin/flock exited; CPU time 11.828434
10:09:21 (14352): app exit status: 0x1
10:09:21 (14352): called boinc_finish(195)

</stderr_txt>
]]>

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid.

To everybody using hyper-threaded CPUs for crunching:
You should test how well the given app scales with HT on or off on your system. The other approach is leave HT on, but lower the percentage of the usable CPUs in BOINC manager (down to 50%). Too many simultaneous memory intensive apps would cause too many cache misses, resulting in degraded combined performance. With HT off (or by setting the usable CPUs to 50%) calculation time should be halved (due that two threads have one FPU). If it's more than a half, then the number of usable CPUs could be increased, while the RAC has risen accordingly (= in a direct ratio).
I can't test it myself until the Windows app has been released, but I'm interested.
A simultaneous GPU task also could degrade the performance of the CPU tasks and vice versa.

I'm trying out the AMD EPYC trial from Packet, it runs 48 QM WUs at a time... all valid.

To everybody using hyper-threaded CPUs for crunching:
You should test how well the given app scales with HT on or off on your system. The other approach is leave HT on, but lower the percentage of the usable CPUs in BOINC manager (down to 50%). Too many simultaneous memory intensive apps would cause too many cache misses, resulting in degraded combined performance. With HT off (or by setting the usable CPUs to 50%) calculation time should be halved (due that two threads have one FPU). If it's more than a half, then the number of usable CPUs could be increased, while the RAC has risen accordingly (= in a direct ratio).
I can't test it myself until the Windows app has been released, but I'm interested.
A simultaneous GPU task also could degrade the performance of the CPU tasks and vice versa.

Zoltan, I think you have a great point here. I am noticing much higher CPU utilization and half of the RAM usage when I switched to 50% CPU in BOINC on these new QC WUs. I think it's mostly to due to the much lower Hard drive bandwidth required and perhaps also the cache on the CPU is more efficiently allocated.

Do you have any tips for getting higher utilization out of these new large molecule QC WUs? I am already running 4 WUs on a 16 core system which is 50% usage in BOINC but the utilization is all over the place. It's using up to 23gb of ram (I have 32gb) with only 4 WUs and I have plenty of space on the SSD.

CPU tasks - unsent: 44,723; in progress: 848; users in last 24hrs: 76

QC WUs are almost out, less than 600 to send out.

They may be waiting until the 3.31 jobs finish before introducing the new 3.32 version. I expect they have plenty more.

Hopefully. We are officially out of cpu work.
____________

I am running two resends. One of them failed with "file too big" error. The other is running.
Tullio

This batch will use lots of scratch space.

OK, the 250 GB SSDs are a good buy at the moment in the U.S.

I submitted some WUs but I am warning you :P This batch will use lots of scratch space.

</stderr_txt>
<message>
upload failure: <file_xfer_error>
<file_name>3320_19_21_22_23_e3641c15_n00001-SDOERR_SELE6-0-1-RND3909_0_1</file_name>
<error_code>-131 (file size too big)</error_code>
</file_xfer_error>

</message>

Two have completed on my Linux box.
Tullio

One more task gained 1848.00 credits.
Tullio

One more task gained 1848.00 credits.
Tullio

Again,1526.20 credits.
Tullio

Again,1526.20 credits.
Tullio

Longer runs don't seem to be affected by the DISK_LIMIT_EXCEEDED error which happens in some shorter runs.My later long run gave me 1151 credits.
Tullio
____________

Run time 3,241.13
CPU time 4,048.97
Validate state Valid
Credit 157.51

Run time 3,241.13
CPU time 4,048.97
Validate state Valid
Credit 157.51

Run time - CPU time - Credit
3,812.94 - 11,088.78 - 2,307.16
4,054.73 - 11,758.62 - 2,637.67

Yes, but his CPU time is much higher, probably because of the core number he has. I have only two.
Tullio

Yes, but his CPU time is much higher, probably because of the core number he has. I have only two.
Tullio

My Linux HP laptop cannot get GPUGRID tasks because it has only 24.90 GB available and the server says it needs 32. So I am running SETI@home on it, which does not require that much space.

Yes the new QMML50s are flowing. Running between 2-4 minutes currently. Will keep an eye on them.
____________

Help! I have no working coming in and it has been a number of days, I see there is plenty of work on the server??? Please advise soonest, thanks Gary
____________

There is GPU work available. And you will need an Nvidia GPU.

There has not been CPU work for a long time.