Before you Come: Computer Setup

Before the workshop, each participant needs to work through the following instructions to ensure their computer is set up to successfully run the workshop materials. Choose which set of instructions to follow based on the operating system of the computer you will be using for the workshop.

Please note, the workshop team does not have a Mac to test these setup instructions, but previous participants with Macs were successful at software installation following steps 2 and 3 of the Windows instructions.

Contents

For Mac Machines

For Windows Machines

Troubleshooting Tips

On the Ceres HPC

For Mac Machines

Please try to troubeshoot installation on your own or with assistance from local IT or other local ARS staff for help. See the end of the Windows section for minimal troubleshooting tips

1) Install Anaconda

If you don’t already have Anaconda installed, follow the instructions for downloading and installing it (for an individual) at https://docs.anaconda.com/anaconda/install/mac-os/.

If you do not have administrative privileges on your machine you may still be able to install to your computer’s Desktop. Otherwise, ask your local IT for assistance installing it. If you do not have local IT assistance, then you can use either the Ceres HPC (if you have an account) or a personal computer.

2-3) Follow the instructions as best you can under “For Windows Machines”

For Windows Machines

Please try to troubeshoot installation on your own or with assistance from local IT or other local ARS staff for help. See the end of this section for inimal troubleshooting tips

1) Install Anaconda

If you don’t already have Anaconda installed, follow the instructions for downloading and installing it (for an individual) at https://docs.anaconda.com/anaconda/install/windows/.

If you do not have administrative privileges on your machine you may still be able to install to your computer’s Desktop. Otherwise, ask your local IT for assistance installing it. If you do not have local IT assistance, then you can use either the Ceres HPC (if you have an account) or a personal computer.

2) Build the workshop Conda environment

From the Windows search bar type “anaconda” and select the Anaconda Powershell Prompt / From the MAC search “terminal” and At the prompt:

conda create --name aiworkshop python=3.8 numpy pandas scipy imageio pillow scikit-learn scikit-image matplotlib hdf5 nodejs jupyterlab -y
  conda activate aiworkshop
  conda install tensorflow
  conda install -c conda-forge opencv
  conda install -c anaconda seaborn
  conda install -c conda-forge prettytable
  conda install -c conda-forge pickle5
  pip install pydot
  python -m ipykernel install --user --name aiworkshop

For installing Graphviz:

For Windows: Download the software from https://graphviz.gitlab.io/download/

For Mac: From terminal

conda activate aiworkshop
brew install graphviz  

When the build finishes, navigate using the Anaconda Powershell Prompt to the folder you want Jupyter notebook to open in (for example create a workshop folder and cd into it) and open Jupyter notebook:

  mkdir MSU-ARS-aiworkshop
  cd MSU-ARS-aiworkshop/
  jupyter lab

3) Run a test Jupyter Notebook and screenshot your results

• launch a new notebook in JupyterLab: File > New > Notebook


• make sure the workshop kernel is selected: Kernel > Change Kernel > select aiworkshop from the drop-down menu


• in the notebook’s empty cell paste this:
  
  import tensorflow as tf
print(tf.__version__)
import cv2
print(cv2.__version__)



• run the cell: click Run > Run All Cells or with your cursor inside the cell type Shift+Enter. The result should tell you you’re using TensorFlow backend and the OpenCV version.


• position the scroll bar so all results can be seen on your screen and then take a screenshot


• paste the screenshot in an email to Dixie Cartwright, dixie@gri.msstate.edu

Troubleshooting Tips

Occasionally, a conda environment build will fail for no apparent reason. Please attempt to build the workshop environment at least 3 times. Sometimes it takes up to 3 attempts, executing the exact same commands for the environment build to complete successfully (no idea why).

If the install of

conda create --name aiworkshop python=3.8 numpy pandas scipy imageio pillow scikit-learn scikit-image matplotlib hdf5 nodejs jupyterlab

completed successfully but you get an error when trying to install tensorflow:

• first try getting tensorflow from the conda-forge channel: replace “conda install tensorflow” with

  conda install -c conda-forge tensorflow
  

• if that doesn’t work, try using pip: replace “conda install tensorflow” with

  pip install tensorflow
  

If an error occurred during the conda environment build process and creation of the environment didn’t complete successfully:

• check if part of the environment was created by typing in the Anaconda Powershell Prompt

    conda env list
  

• If you see ‘aiworkshop’ in the list delete it by typing

    conda env remove --name aiworkshop
  

  and start at the beginning of ‘Step 2 Build the workshop Conda environment’ i again. Try this process at least 3 times before giving up.


• If you don’t see ‘aiworkshop’ in the list then you don’t need to delete anything, just try step 2 of the installation again. Try this process at least 3 times before giving up.

If your environment builds successfully but ‘aiworkshop’ doesn’t appear as a kernel option in jupyter lab:

• first try relauching Jupyter Lab
  • in Jupyter Lab: File > Shut Down
  • in Anaconda Powershell Prompt: jupyter lab
  • in Jupyter Lab: try selecting the kernel again


• if that doesn’t work shut down Jupyter Lab, delete the whole environment and start over. In the Anaconda Powershell Prompt type

    conda env remove --name aiworkshop
  

  and start over with ‘Step 2 Build the workshop Conda environment’ again. Try this process at least 3 times before giving up.

On the Ceres HPC

Please try to troubeshoot installation on your own or contact HPC support at scinet_vrsc@usda.gov

1) Request a project directory on Ceres if you don’t already have one or use /90daydata/scinet/yourname/

You will not be able to run the workshop materials in your HPC home directory due to the space limitations. If you already have a project directory, proceed to step 2- you can run the workshop materials from any existing project directory with approximately 20GB of free space.

If you do not have a project directory yet, you can request one using the project directory request form. eAuth is required to access the form. If you do not have eAuth credentials (i.e. you don’t have a USDA PIV or CAC card) you will need to ask your USDA sponsor to complete the project directory request for you. Do this quickly since the approval process can take a week or more. Make sure to state on the request form that you need the directory to participate in a SCINet training event and give them the workshop start date.

The other option is to use the /90daydata/scinet/ folder. As the folder name suggests, files stored there are not permanent and will be deleted after 90 days. We suggest creating a new folder at /90daydata/scinet/yourname/, building the workshop conda environment there, and also running the workshop jupyter notebooks from that location as well. To do this you would modify the instructions below by substituting /90daydata/scinet/yourname/ wherever you see /project/your_project_name/.

Log into the Ceres HPC using JupyterHub

• Go to https://jupyterhub.scinet.usda.gov/


• Use your SCINet credentials to log in.
  • Username: firstname.lastname
  • Verification Code: 6 digit time-sensitive code that comes from Google Authenticator
  • Password: your SCINet password


• Enter the following info on the spawner page
  • Node Type: short
  • Number of Cores: 4
  • Job Duration: 02:00:00
  • Working Directory: /lustre/project/your_project_name or /90daydata/scinet
  • leave all other fields blank

3) Build the workshop Conda environment

If using /90daydata/scinet, create a new folder there mkdir /90daydata/scinet/yourname, then modify the instructions below by substituting /90daydata/scinet/yourname/ wherever you see /project/your_project_name/.

• open a terminal in JupyterLab with File > New > Terminal


• navigate to your project directory

    cd /project/your_project_name

• download the workshop yml file

    wget https://kerriegeil.github.io/NMSU-USDA-ARS-AI-Workshops/aiworkshop.yml

• build the environment in your project directory

    source activate
    conda env create --prefix /project/your_project_name/envs/aiworkshop -f aiworkshop.yml
  

  The build may take a while- up to 10 minutes or possibly longer. When the build finishes:

    conda activate /project/your_project_name/envs/aiworkshop

4) Run a test Jupyter Notebook and screen shot your results

• launch a new notebook in JupyterLab: File > New > Notebook


• make sure the workshop kernel is selected: Kernel > Change Kernel > select aiworkshop from the drop-down menu


• in the notebook’s empty cell paste this:
  
  import tensorflow as tf
print(tf.__version__)
import cv2
print(cv2.__version__)


• run the cell: click Run > Run All Cells or with your cursor inside the cell type Shift+Enter. The result should tell you you’re using TensorFlow backend and the OpenCV version.


• position the scroll bar so all results can be seen on your screen and then take a screenshot


• paste the screenshot in an email to Dixie Cartwright, dixie@gri.msstate.edu