Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CUDA error: an illegal memory access was encountered #2

Open
tjusxh opened this issue May 19, 2020 · 0 comments
Open

CUDA error: an illegal memory access was encountered #2

tjusxh opened this issue May 19, 2020 · 0 comments

Comments

@tjusxh
Copy link

tjusxh commented May 19, 2020

I implement the vae (gcn) according to your code. But when I run the code, I encounter the following error in the second iteration. Do you encounter the problem?

step0 tensor(0, device='cuda:0')
step1 tensor(0, device='cuda:0')
step2 Traceback (most recent call last):
File "train.py", line 235, in
train_loss = train(epoch)
File "train.py", line 156, in train
z = model.encode(x, edge_idxs)
File "/root/hair_vae/vgae/model/vgae.py", line 108, in encode
self.mu, self.logvar = self.encoder(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/root/hair_vae/vgae/model/vgae.py", line 44, in forward
print("step2", torch.isnan(x).sum())
File "/usr/local/lib/python3.6/dist-packages/torch/tensor.py", line 162, in repr
return torch._tensor_str._str(self)
File "/usr/local/lib/python3.6/dist-packages/torch/_tensor_str.py", line 315, in _str
tensor_str = _tensor_str(self, indent)
File "/usr/local/lib/python3.6/dist-packages/torch/_tensor_str.py", line 213, in _tensor_str
formatter = _Formatter(get_summarized_data(self) if summarize else self)
File "/usr/local/lib/python3.6/dist-packages/torch/_tensor_str.py", line 84, in init
value_str = '{}'.format(value)
File "/usr/local/lib/python3.6/dist-packages/torch/tensor.py", line 418, in format
return self.item().format(format_spec)
RuntimeError: CUDA error: an illegal memory access was encountered

x = F.elu(self.conv1(x, edge_indexs[0]))
print("step0", torch.isnan(x).sum())
x = Pool(x, self.down_samples[0])
print("step1", torch.isnan(x).sum())
x= self.conv2(x, edge_indexs[1])
print("step2", torch.isnan(x).sum())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant