Showing posts with label python. Show all posts
Showing posts with label python. Show all posts

Python contextlib for Timing Python code

If you've ever found yourself needing to measure the execution time of specific portions of your Python code, the `contextlib` module offers an elegant solution. In particular, we can use it to create a timer that functions as a context manager, simplifying the process of tracking performance without cluttering your code with timing logic. 

Consider the following example of a timer implemented as a context manager: 

from contextlib import contextmanager
import time

@contextmanager
def timer(label):
    start = time.time()
    try:
        yield
    finally:
        end = time.time()
        print(f'{label}: {end - start} seconds')

# Usage:
with timer('Model training'):
    # Your code block here, e.g., train your model

In this example, the `timer` function is decorated with `@contextmanager`, transforming it into a context manager. When you use this timer within a `with` statement, it automatically measures the time taken to execute the enclosed code block. 

This feature proves exceptionally useful for profiling the performance of various parts of your codebase. For instance, in the context of machine learning, you can easily measure the duration of model training or inference without inserting cumbersome timing statements throughout your code. 

By adopting Python's context manager and the `contextlib` module, you can streamline the process of performance profiling, making your code more readable and focused on its primary logic. Try incorporating this timer into your projects to gain valuable insights into the execution time of critical code segments.

Get PIP package versions and licenses within a virtual environment

pip install pip-licenses

pip-licenses --format=csv \
--with-urls \
--with-authors \
--with-maintainers \
--with-description > packages.csv

# The following will produce main packages only without nested dependencies.
pip install pipdeptree
pipdeptree -f | grep -P '^\w+' | cut -d = -f 1 > main-packages.txt

Build OpenVINO from source - Linux

Official Instructions - Click Here

Build instructions for OpenVINO from Source with the Python API Wrapper

OpenVINO 2022.1.0 and later require GLIBC 2.27+, check with `ldd --version`

Here instructions are for Python 3.7, you can change it to 3.6 as well.

Software Requirements:

- CMake 3.13 or higher
- GCC 7.5 or higher to build OpenVINO Runtime
- Python 3.6 or higher for OpenVINO Runtime Python API
- (Optional) Install Intel® Graphics Compute Runtime for OpenCL™ Driver package 19.41.14441 to enable inference on Intel integrated GPUs.


$ sudo apt-get install python3.7-dev

$ pip install cython numpy

$ cd ~ #openvino will be installed in ~/openvino

$ git clone https://github.com/openvinotoolkit/openvino.git
$ cd openvino
$ git submodule update --init --recursive
$ chmod +x install_dependencies.sh
$ ./install_dependencies.sh
$ mkdir build && cd build

$ cmake -DCMAKE_BUILD_TYPE=Release \
-DENABLE_INTEL_GNA=OFF -DENABLE_INTEL_MYRIAD_COMMON=OFF \
-DENABLE_PYTHON=ON \
-DPYTHON_EXECUTABLE=`which python3.7` \
-DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.7m.so \
-DPYTHON_INCLUDE_DIR=/usr/include/python3.7 ..

$ make --jobs=$(nproc --all)

$ export PYTHONPATH=$PYTHONPATH:~/openvino/bin/intel64/Release/lib/python_api/python3.7/
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/openvino/bin/intel64/Release/lib/

# OR Install the wheel with PIP
$ pip install ~/openvino/build/wheel/*.whl

# TEST BUILD
$ python3.7
>>> from openvino.inference_engine import IENetwork, IECore

# Test Benchmark app

$ alias benchmark_app=~/openvino/bin/intel64/Release/benchmark_app 
$ benchmark_app -h

Install Python3.6 or Python3.7 in Ubuntu 20.04; ImportError: libpython3.6m.so.1.0: cannot open shared object file

Ubuntu 20.04 has Python 3.8 by default. 
See below to install Python 3.6 or 3.7

sudo -E add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.6
sudo apt-get install -y libpython3.6-dev

Install like above if you get below error

ImportError: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory


Debugging Python with pudb

1. Install pudb pip package..

 $ pip install pudb

2. Add the following statement in python code.
import pudb; pudb.set_trace()
3. Or, Launch the debugger like below
$ python -m pudb my_script.py

Manage logging in python. Suppress and Set Log Level in Python

Example to supress Tensorflow logging

import logging
logging.getLogger('tensorflow').setLevel(logging.ERROR)

To get all the list of available loggers:

import logging
for key in logging.Logger.manager.loggerDict:
    print(key)

Python contextlib for Timing Python code

If you've ever found yourself needing to measure the execution time of specific portions of your Python code, the `contextlib` module o...