Manage logging in python. Suppress and Set Log Level in Python

Example to supress Tensorflow logging

import logging
logging.getLogger('tensorflow').setLevel(logging.ERROR)

To get all the list of available loggers:

import logging
for key in logging.Logger.manager.loggerDict:
    print(key)

Check if MKL is enabled in Tensorflow installation

Check if MKL is enabled in Tensorflow

python -c "import tensorflow; print(tensorflow.pywrap_tensorflow.IsMklEnabled())"

Start Jupyter or Jupyter-lab on startup of ec-2 instance.

Start Jupyter or Jupyter-lab on startup of ec-2 instance.

Edit the crontab by running

sudo crontab -e

and add the following

@reboot su ec2-user -c "/home/ec2-user/.start_jupyter.sh"

Create a file 

vi /home/ec2-user/.start_jupyter.sh

and add the following

cd /home/ec2-user

source /home/ec2-user/.bashrc

source /opt/intel/openvino/bin/setupvars.sh

/home/ec2-user/.local/bin/jupyter notebook --no-browser --NotebookApp.allow_password_change=False --NotebookApp.token="$INSTANCE_ID" --ip 0.0.0.0 --port 8888 > /tmp/jupyter.out 2>&1 &

We can leave this option to true, but changing password is not working...

--NotebookApp.allow_password_change=False because change password is not working. It is a bug in Jupyter. See https://github.com/jupyter/notebook/issues/3842

Give executable file permissions.

chmod 755 .start_jupyter.sh 

Debugging

sudo cat /var/log/cron

Also,

cat /tmp/jupyter.out 

If you want to set password randomly on EC2-Instance:

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`

INSTANCE_ID=`curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id`

cd /home/ec2-user

source /home/ec2-user/.bashrc

source /opt/intel/openvino/bin/setupvars.sh

/home/ec2-user/.local/bin/jupyter notebook --no-browser --NotebookApp.allow_password_change=False --NotebookApp.token="$INSTANCE_ID" --ip 0.0.0.0 --port 8888 > /tmp/jupyter.out 2>&1 &

See:

https://docs.aws.amazon.com/marketplace/latest/userguide/product-and-ami-policies.html#security

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

Set Python3 as default in CentOS or Amazon Linux2

# Install Python3
sudo yum install -y python3

# Start by registering python2 as an alternative
sudo alternatives --install /usr/bin/python python /usr/bin/python2 50

# Register python3.5 as an alternative
sudo alternatives --install /usr/bin/python python /usr/bin/python3.5 60

# Select which Python version to use
sudo alternatives --config python

Ubuntu 18.04: Create/Modify SWAP partition memory limit.

Common error:

fallocate: fallocate failed: Text file busy in ubuntu OS 

Check the allocation:

free -m
or
sudo swapon --show

Make sure you have enough free disk space by

df -h

Disable the use of swap, allocate new swap space, enable swap.

sudo swapoff -a

sudo fallocate -l 8G /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

Check the allocation:

free -m
or
sudo swapon --show

For detailed information: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-18-04

Connecting to Ubuntu Remote Desktop from Mac OS

Connecting to Ubuntu or any linux server via Mac OS:
-- Access the "Spotlight" by hitting CMD + Space-bar
-- Within the Spotlight field, enter vnc://your_server_ip:5900

If you get an error when connecting to Ubuntu from Mac OS using VNC screen sharing:
"The software on the remote computer appears to be incompatible with this version of Screen Sharing."
On Ubuntu 18.04:

$ gsettings set org.gnome.Vino require-encryption false

Verify the setting with

$ gsettings list-recursively org.gnome.Vino | grep encrypt

Fix wrong time in Ubuntu

Fix wrong time in Ubuntu .

sudo timedatectl set-local-rtc 1 --adjust-system-clock

BIOS settings to Install Ubuntu

If Ubuntu installation shows only the USB drive from which you are installing, make sure you have the BIOS settings as follows.

BIOS settings to Install Ubuntu alongside with Windows :

1. Disable Secure Boot.
2. Set SATA-controller to AHCI from RAID On.
3. Set boot mode to legacy from UEFI.

How to fix error Requires: libva.so.1(VA_API_0.33.0)(64bit)

If you get the following error while installing FFMPEG or other packages on CentOS 7.

Requires: libva.so.1(VA_API_0.33.0)(64bit)

Follow the following steps to install libva

sudo yum install libwayland-client    

wget http://mirror.centos.org/centos/7/os/x86_64/Packages/libva-1.8.3-1.el7.x86_64.rpm 

sudo rpm -i libva-1.8.3-1.el7.x86_64.rpm          

Sample error log when installing FFMPEG:

Error: Package: ffmpeg-libs-3.4.7-1.el7.x86_64 (rpmfusion-free-updates)
           Requires: libva.so.1(VA_API_0.33.0)(64bit)
Error: Package: ffmpeg-libs-3.4.7-1.el7.x86_64 (rpmfusion-free-updates)
           Requires: libva-x11.so.1()(64bit)
Error: Package: libmfx-1.21-2.el7.x86_64 (epel)
           Requires: libva-x11.so.1()(64bit)
Error: Package: libmfx-1.21-2.el7.x86_64 (epel)
           Requires: libva-drm.so.1()(64bit)
Error: Package: ffmpeg-libs-3.4.7-1.el7.x86_64 (rpmfusion-free-updates)
           Requires: libva-drm.so.1()(64bit)
Error: Package: ffmpeg-libs-3.4.7-1.el7.x86_64 (rpmfusion-free-updates)
           Requires: libva.so.1()(64bit)
Error: Package: libmfx-1.21-2.el7.x86_64 (epel)
           Requires: libva.so.1()(64bit)