self (deep-gpu-1)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- -------- ------------ ------ ----------------------------
0 [~] GeForce GTX 1080 Ti 0 % 3.66GB 29 139 engstrom 20 hours ago 37994 python -m src.main --lr 0...
1 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 42 1480
2 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 49 1480
3 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 44 1480
4 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 46 1480
5 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 40 1480
6 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 45 1480
7 [ ] GeForce GTX 1080 Ti 5 % 0.0GB 45 227
self (deep-gpu-2)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- ---------- --------- ------ ----------------------
0 [~] GeForce GTX 1080 Ti 19 % 11.46GB 70 1847 siddhartha a day ago 24523 train_breeds_marked.py
1 [~] GeForce GTX 1080 Ti 30 % 11.31GB 76 1822 siddhartha a day ago 24523 train_breeds_marked.py
2 [~] GeForce GTX 1080 Ti 94 % 11.46GB 82 1835 siddhartha a day ago 27201 train_breeds_marked.py
3 [~] GeForce GTX 1080 Ti 100 % 11.31GB 78 1809 siddhartha a day ago 27201 train_breeds_marked.py
4 [~] GeForce GTX 1080 Ti 100 % 11.46GB 83 1835 siddhartha a day ago 27822 train_breeds_marked.py
5 [~] GeForce GTX 1080 Ti 100 % 11.31GB 79 1847 siddhartha a day ago 27822 train_breeds_marked.py
6 [~] GeForce GTX 1080 Ti 100 % 11.46GB 80 1835 siddhartha a day ago 28422 train_breeds_marked.py
7 [~] GeForce GTX 1080 Ti 100 % 11.31GB 77 1822 siddhartha a day ago 28422 train_breeds_marked.py
self (deep-gpu-3)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- ---------- ----------- ---------- -------------------
0 [~] GeForce GTX 1080 Ti 0 % 6.74GB 29 139 ?,? 35226,7924
1 [!] GeForce GTX 1080 Ti 0 % 0.73GB 52 1480 ? 7924
2 [!] GeForce GTX 1080 Ti 0 % 0.73GB 29 139 ? 7924
3 [~] GeForce GTX 1080 Ti 88 % 10.6GB 84 1759 ?,wongeric a day ago 7924,16896 robust_train.py
4 [~] GeForce GTX 1080 Ti 86 % 8.62GB 85 1771 ?,wongeric 7 hours ago 7924,33358 imagenet_example.py
5 [~] GeForce GTX 1080 Ti 0 % 3.91GB 62 1885 ?,wongeric a day ago 7924,16896 robust_train.py
6 [~] GeForce GTX 1080 Ti 47 % 3.91GB 71 1860 ?,wongeric a day ago 7924,16896 robust_train.py
7 [~] GeForce GTX 1080 Ti 10 % 3.91GB 68 1860 ?,wongeric a day ago 7924,16896 robust_train.py
self (deep-gpu-4)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- ------- ----------- ------ -------
0 [~] GeForce GTX 1080 Ti 0 % 2.99GB 67 1847 kaveri an hour ago 36165 main.py
1 [~] GeForce GTX 1080 Ti 0 % 2.91GB 72 1847 kaveri an hour ago 36165 main.py
2 [~] GeForce GTX 1080 Ti 0 % 7.58GB 83 1771 kaveri 4 hours ago 28020 main.py
3 [~] GeForce GTX 1080 Ti 0 % 7.48GB 78 1835 kaveri 4 hours ago 28020 main.py
4 [~] GeForce GTX 1080 Ti 99 % 7.58GB 77 1771 kaveri an hour ago 36665 main.py
5 [~] GeForce GTX 1080 Ti 95 % 7.48GB 69 1771 kaveri an hour ago 36665 main.py
6 [~] GeForce GTX 1080 Ti 99 % 7.58GB 83 1771 kaveri 4 hours ago 28717 main.py
7 [~] GeForce GTX 1080 Ti 99 % 7.48GB 85 1480 kaveri 4 hours ago 28717 main.py
self (deep-gpu-5)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- ------------------------ -------------- ---------------- -------------------
0 [~] GeForce GTX 1080 Ti 45 % 11.26GB 65 1847 spark 22 minutes ago 34408 eval_corruptions.py
1 [~] GeForce GTX 1080 Ti 35 % 10.67GB 74 1797 spark 22 minutes ago 34408 eval_corruptions.py
2 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 31 139
3 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 32 139
4 [~] GeForce GTX 1080 Ti 72 % 9.88GB 83 1784 wongeric a day ago 18985 robust_train.py
5 [~] GeForce GTX 1080 Ti 0 % 3.19GB 62 1885 wongeric a day ago 18985 robust_train.py
6 [~] GeForce GTX 1080 Ti 11 % 3.19GB 69 1822 wongeric a day ago 18985 robust_train.py
7 [~] GeForce GTX 1080 Ti 27 % 6.42GB 65 1835 wongeric,tsipras,tsipras a day ago 18985,3888,34244 robust_train.py
self (deep-gpu-6)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- ------- ------------ ------ -------
0 [~] GeForce GTX 1080 Ti 0 % 6.51GB 31 139 shibani 14 hours ago 31811 jupyter
1 [~] GeForce GTX 1080 Ti 0 % 6.99GB 31 139 keqian 2 weeks ago 21612
2 [~] GeForce GTX 1080 Ti 0 % 1.53GB 36 139 shibani 2 days ago 36545 jupyter
3 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 41 1506
4 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 39 139
5 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 43 1506
6 [ ] GeForce GTX 1080 Ti 6 % 0.0GB 39 139
7 [~] GeForce GTX 1080 Ti 0 % 6.6GB 31 139 shibani 11 hours ago 34708 jupyter
self (deep-gpu-7)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- -------- ----------- ------ -----------------------
0 [~] GeForce GTX 1080 Ti 100 % 9.88GB 78 1759 wongeric a day ago 32487 robust_train.py
1 [ ] GeForce GTX 1080 Ti 0 % 0.0GB 39 227
2 [~] GeForce GTX 1080 Ti 34 % 3.19GB 69 1898 wongeric a day ago 32487 robust_train.py
3 [~] GeForce GTX 1080 Ti 23 % 3.19GB 62 1898 wongeric a day ago 32487 robust_train.py
4 [~] GeForce GTX 1080 Ti 0 % 11.14GB 28 139 tsipras a month ago 10742 python - GPU HAS ISSUES
5 [~] GeForce GTX 1080 Ti 0 % 3.19GB 57 1885 wongeric a day ago 32487 robust_train.py
6 [~] GeForce GTX 1080 Ti 0 % 1.09GB 29 139 tsipras 2 weeks ago 34638 jupyter
7 [~] GeForce GTX 1080 Ti 0 % 1.18GB 32 139 tsipras a month ago 5962 jupyter
self (deep-gpu-8)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- ---------- -------------- ------ ----------------------
0 [~] GeForce GTX 1080 Ti 98 % 11.44GB 80 1784 siddhartha a day ago 13000 train_breeds_marked.py
1 [~] GeForce GTX 1080 Ti 0 % 7.06GB 30 139 keqian 3 days ago 17749 jupyter
2 [~] GeForce GTX 1080 Ti 67 % 8.77GB 78 1898 wongeric 15 minutes ago 31376 cifar10_transfer.py
3 [ ] GeForce GTX 1080 Ti 3 % 0.0GB 40 1506
4 [~] GeForce GTX 1080 Ti 95 % 9.88GB 81 1784 wongeric a day ago 34638 robust_train.py
5 [~] GeForce GTX 1080 Ti 21 % 3.19GB 55 1911 wongeric a day ago 34638 robust_train.py
6 [~] GeForce GTX 1080 Ti 18 % 3.19GB 65 1898 wongeric a day ago 34638 robust_train.py
7 [~] GeForce GTX 1080 Ti 34 % 3.19GB 63 1898 wongeric a day ago 34638 robust_train.py
self (deep-gpu-9)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- --------------- ------------- ------------- ---------------------------------------------------------------------------------
0 [~] GeForce RTX 2080 Ti 41 % 7.99GB 68 1860 aiilyas,aiilyas 6 minutes ago 121655,121654 /data/theory/robustopt/andrew/subpopulations/src/mem_scores/train_cifar_subset.py
1 [~] GeForce RTX 2080 Ti 63 % 7.99GB 72 1755 aiilyas,aiilyas 6 minutes ago 121657,121656 /data/theory/robustopt/andrew/subpopulations/src/mem_scores/train_cifar_subset.py
2 [~] GeForce RTX 2080 Ti 42 % 7.99GB 72 1830 aiilyas,aiilyas 6 minutes ago 121659,121658 /data/theory/robustopt/andrew/subpopulations/src/mem_scores/train_cifar_subset.py
3 [~] GeForce RTX 2080 Ti 79 % 7.99GB 72 1770 aiilyas,aiilyas 6 minutes ago 121661,121660 /data/theory/robustopt/andrew/subpopulations/src/mem_scores/train_cifar_subset.py
4 [~] GeForce RTX 2080 Ti 48 % 7.99GB 68 1875 aiilyas,aiilyas 6 minutes ago 121662,121663 /data/theory/robustopt/andrew/subpopulations/src/mem_scores/train_cifar_subset.py
5 [~] GeForce RTX 2080 Ti 60 % 7.99GB 74 1800 aiilyas,aiilyas 6 minutes ago 121665,121664 /data/theory/robustopt/andrew/subpopulations/src/mem_scores/train_cifar_subset.py
6 [~] GeForce RTX 2080 Ti 73 % 7.99GB 72 1830 aiilyas,aiilyas 6 minutes ago 121666,121667 /data/theory/robustopt/andrew/subpopulations/src/mem_scores/train_cifar_subset.py
7 [~] GeForce RTX 2080 Ti 53 % 7.99GB 68 1770 aiilyas,aiilyas 6 minutes ago 121668,121669 /data/theory/robustopt/andrew/subpopulations/src/mem_scores/train_cifar_subset.py
self (deep-gpu-10)
status type util. mem. used temp. MHz users since pids cmd
-- -------- ------------------- ------- ----------- ------- ----- ------- ------------ ------ ----------------------------
0 [~] GeForce RTX 2080 Ti 27 % 11.39GB 66 1680 manishb 15 hours ago 436803 /opt/miniconda/bin/python...
1 [~] GeForce RTX 2080 Ti 36 % 11.39GB 69 1710 manishb 15 hours ago 436804 /opt/miniconda/bin/python...
2 [~] GeForce RTX 2080 Ti 98 % 11.39GB 71 1755 manishb 15 hours ago 436805 /opt/miniconda/bin/python...
3 [~] GeForce RTX 2080 Ti 98 % 11.39GB 67 1785 manishb 15 hours ago 436806 /opt/miniconda/bin/python...
4 [ ] GeForce RTX 2080 Ti 0 % 0.0GB 38 1350
5 [ ] GeForce RTX 2080 Ti 0 % 0.0GB 37 1350
6 [ ] GeForce RTX 2080 Ti 1 % 0.0GB 40 1350
7 [ ] GeForce RTX 2080 Ti 0 % 0.0GB 39 1350