I followed this tutorial and installed tf-gpu using conda (https://www.pugetsystems.com/labs/hpc/The-Best-Way-to-Install-TensorFlow-with-GPU-Support-on-Windows-10-Without-Installing-CUDA-1187/) and it worked because I am seeing "...gpu:0" in my printed out log. Before I did the installation, I already have CUDA driver installed, so I am not sure.
Seems to me that conda install tensorflow-gpu comes with cuda toolkit and cuDNN,etc. I was wondering if installing CUDA driver is a require step. Another post I found did't mention driver either (https://towardsdatascience.com/tensorflow-gpu-installation-made-easy-use-conda-instead-of-pip-52e5249374bc). But the official GPU guide says it's required, so I am confused. I am doing it on Windows 10.
conda install cuda
does not install a GPU driver. and a GPU driver is required to use a CUDA GPU. On windows, in many cases a GPU driver will be automatically installed by windows, e.g. for typical WDDM GPUs like GeForce GPUs. Whether or not this driver is compatible with the toolkit version installed by conda is perhaps not guaranteed, but I think it generally would be compatible.