Install iGPU / Intel integrated GPU drivers for OpenVINO Inference

To enable Intel integrated GPU / iGPU for running deep learning inference with OpenVINO, we need to install the Intel Graphics Compute Runtime for oneAPI Level Zero and OpenCL™ Driver on Linux.

Intel Graphics Compute Runtime:  https://github.com/intel/compute-runtime/releases

See : https://docs.openvino.ai/2024/get-started/configurations/configurations-intel-gpu.html

More info: https://dgpu-docs.intel.com/driver/client/overview.html

# After Installing Intel Graphics Compute Runtime

sudo apt install hwinfo clinfo -y

After installation, add video and render user groups.

sudo usermod -aG video $USER
sudo usermod -aG render $USER

# Logout and login 

Build OpenVINO from source - Linux

Official Instructions - Click Here

Build instructions for OpenVINO from Source with the Python API Wrapper

OpenVINO 2022.1.0 and later require GLIBC 2.27+, check with `ldd --version`

Here instructions are for Python 3.7, you can change it to 3.6 as well.

Software Requirements:

- CMake 3.13 or higher
- GCC 7.5 or higher to build OpenVINO Runtime
- Python 3.6 or higher for OpenVINO Runtime Python API
- (Optional) Install Intel® Graphics Compute Runtime for OpenCL™ Driver package 19.41.14441 to enable inference on Intel integrated GPUs.

$ sudo apt-get install python3.7-dev

$ pip install cython numpy

$ cd ~ #openvino will be installed in ~/openvino

$ git clone https://github.com/openvinotoolkit/openvino.git
$ cd openvino
$ git submodule update --init --recursive
$ chmod +x install_dependencies.sh
$ ./install_dependencies.sh
$ mkdir build && cd build

$ cmake -DCMAKE_BUILD_TYPE=Release \

-DENABLE_INTEL_GNA=OFF -DENABLE_INTEL_MYRIAD_COMMON=OFF \

-DENABLE_PYTHON=ON \
-DPYTHON_EXECUTABLE=`which python3.7` \
-DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.7m.so \
-DPYTHON_INCLUDE_DIR=/usr/include/python3.7 ..

$ make --jobs=$(nproc --all)

$ export PYTHONPATH=$PYTHONPATH:~/openvino/bin/intel64/Release/lib/python_api/python3.7/
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/openvino/bin/intel64/Release/lib/

# OR Install the wheel with PIP

$ pip install ~/openvino/build/wheel/*.whl

# TEST BUILD

$ python3.7
>>> from openvino.inference_engine import IENetwork, IECore

# Test Benchmark app

$ alias benchmark_app=~/openvino/bin/intel64/Release/benchmark_app 
$ benchmark_app -h

Install Python3.6 or Python3.7 in Ubuntu 20.04; ImportError: libpython3.6m.so.1.0: cannot open shared object file

Ubuntu 20.04 has Python 3.8 by default. 
See below to install Python 3.6 or 3.7

sudo -E add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.6
sudo apt-get install -y libpython3.6-dev

Install like above if you get below error

ImportError: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory

Debugging Python with pudb

1. Install pudb pip package..

 $ pip install pudb

2. Add the following statement in python code.

import pudb; pudb.set_trace()

3. Or, Launch the debugger like below

$ python -m pudb my_script.py

TMUX Scrolling with Mouse

TMUX Scrolling with Mouse

# Add these in ~/.tmux.conf

set -g mouse on    #For tmux version 2.1 and up  
set -g @scroll-down-exit-copy-mode "off"
set -g terminal-overrides 'xterm*:smcup@:rmcup@' # Sane scrolling

# RELOAD the conf file:  

# tmux source-file ~/.tmux.conf

# If you get Error: protocol version mismatch (client 8, server 7)

# sudo killall -9 tmux

# If conf error, make sure you have tmux version >2.1

# tmux -V

# Update tmux. Ex: sudo yum install tmux

Install ClamAV on Amazon Linux 2

Install ClamAV Scan on Amazon Linux 2

sudo amazon-linux-extras install epel
sudo yum install clamav clamd -y
sudo sed -i -e "s/Example/#Example/" /etc/freshclam.conf
sudo sed -i -e "s:#DatabaseDirectory /var/lib/clamav:DatabaseDirectory /var/lib/clamav:" /etc/freshclam.conf
sudo sed -i -e "s:#UpdateLogFile /var/log/freshclam.log:UpdateLogFile /var/log/freshclam.log:" /etc/freshclam.conf
sudo sed -i -e "s/#DatabaseOwner clamupdate/DatabaseOwner clamupdate/" /etc/freshclam.conf
sudo vi /etc/clamd.d/scan.conf 
# Change MaxThreads if desired.

sudo freshclam
clamscan -r / 2>&1 | tee openvinoami_clamav_scan.txt

# If problems arise, restart
sudo pkill freshclam
sudo freshclam
clamscan -r / 2>&1 | tee openvinoami_clamav_scan.txt

Dynamic Port Forwarding using Firefox Browser

[bob@workstation ~]$ ssh -D 1080 bastion.securecorp.io [bob@bastion ~]$

SOCKS5 configuration in Firefox:

Source: https://www.redhat.com/sysadmin/ssh-dynamic-port-forwarding

Host a static website in Ubuntu using Nginx

 Install Ngnix Web Server

sudo apt-get install nginx

Controlling Nginx

sudo systemctl start nginx

sudo systemctl stop nginx

sudo systemctl restart nginx

sudo systemctl reload nginx

Firewall

sudo ufw allow 'Nginx Full'

sudo ufw reload

Nginx Configuration Directory

cd /etc/nginx

sudo cp nginx.conf nginx.conf.bak

sudo nano nginx.conf

Verify this line is uncommented, include /etc/nginx/sites-enabled/*;




cd sites-available

sudo nano default




server { listen 80 default_server; listen [::]:80 default_server; server_name _; root /home/ravi/www/html; index index.html; location / { # First attempt to serve request as file, then # as directory, then fall back to displaying a 404. try_files $uri $uri/ =404; } }

Restart Server

sudo systemctl restart nginx

Source: https://medium.com/@jasonrigden/how-to-host-a-static-website-with-nginx-8b2dd0c5b301

Manage logging in python. Suppress and Set Log Level in Python

Example to supress Tensorflow logging

import logging
logging.getLogger('tensorflow').setLevel(logging.ERROR)

To get all the list of available loggers:

import logging
for key in logging.Logger.manager.loggerDict:
    print(key)

Check if MKL is enabled in Tensorflow installation

Check if MKL is enabled in Tensorflow

python -c "import tensorflow; print(tensorflow.pywrap_tensorflow.IsMklEnabled())"

Start Jupyter or Jupyter-lab on startup of ec-2 instance.

Start Jupyter or Jupyter-lab on startup of ec-2 instance.

Edit the crontab by running

sudo crontab -e

and add the following

@reboot su ec2-user -c "/home/ec2-user/.start_jupyter.sh"

Create a file 

vi /home/ec2-user/.start_jupyter.sh

and add the following

cd /home/ec2-user

source /home/ec2-user/.bashrc

source /opt/intel/openvino/bin/setupvars.sh

/home/ec2-user/.local/bin/jupyter notebook --no-browser --NotebookApp.allow_password_change=False --NotebookApp.token="$INSTANCE_ID" --ip 0.0.0.0 --port 8888 > /tmp/jupyter.out 2>&1 &

We can leave this option to true, but changing password is not working...

--NotebookApp.allow_password_change=False because change password is not working. It is a bug in Jupyter. See https://github.com/jupyter/notebook/issues/3842

Give executable file permissions.

chmod 755 .start_jupyter.sh 

Debugging

sudo cat /var/log/cron

Also,

cat /tmp/jupyter.out 

If you want to set password randomly on EC2-Instance:

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`

INSTANCE_ID=`curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id`

cd /home/ec2-user

source /home/ec2-user/.bashrc

source /opt/intel/openvino/bin/setupvars.sh

/home/ec2-user/.local/bin/jupyter notebook --no-browser --NotebookApp.allow_password_change=False --NotebookApp.token="$INSTANCE_ID" --ip 0.0.0.0 --port 8888 > /tmp/jupyter.out 2>&1 &

See:

https://docs.aws.amazon.com/marketplace/latest/userguide/product-and-ami-policies.html#security

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

Set Python3 as default in CentOS or Amazon Linux2

# Install Python3
sudo yum install -y python3

# Start by registering python2 as an alternative
sudo alternatives --install /usr/bin/python python /usr/bin/python2 50

# Register python3.5 as an alternative
sudo alternatives --install /usr/bin/python python /usr/bin/python3.5 60

# Select which Python version to use
sudo alternatives --config python

Ubuntu 18.04: Create/Modify SWAP partition memory limit.

Common error:

fallocate: fallocate failed: Text file busy in ubuntu OS 

Check the allocation:

free -m
or
sudo swapon --show

Make sure you have enough free disk space by

df -h

Disable the use of swap, allocate new swap space, enable swap.

sudo swapoff -a

sudo fallocate -l 8G /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

Check the allocation:

free -m
or
sudo swapon --show

For detailed information: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-18-04

Connecting to Ubuntu Remote Desktop from Mac OS

Connecting to Ubuntu or any linux server via Mac OS:
-- Access the "Spotlight" by hitting CMD + Space-bar
-- Within the Spotlight field, enter vnc://your_server_ip:5900

If you get an error when connecting to Ubuntu from Mac OS using VNC screen sharing:
"The software on the remote computer appears to be incompatible with this version of Screen Sharing."
On Ubuntu 18.04:

$ gsettings set org.gnome.Vino require-encryption false

Verify the setting with

$ gsettings list-recursively org.gnome.Vino | grep encrypt

Fix wrong time in Ubuntu

Fix wrong time in Ubuntu .

sudo timedatectl set-local-rtc 1 --adjust-system-clock