๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Horace He
@cHHillee
@thinkymachines Formerly @PyTorch "My learning style is Horace twitter threads" - @typedfemale
๊ฐ€์ž… February 2010
633 ํŒ”๋กœ์ž‰ ์ค‘    55.4K ํŒฌ
Feels like it's been some time since I've learned something new about non-determinism on GPUs ๐Ÿ˜† Pretty cool!
1/4 New writeup: you can fingerprint the exact GPU you're running on with a default torch.index_add_ call, even among identical GPUs. This works because floating-point maths depends on order, and every GPU has its own. This fingerprint survives reboots and can't be faked.
๋” ๋ณด๊ธฐ