Advertisements

Packaging & Hosting python repo to S3

Reading Time: 2 minutes

This blog walks through the packaging of simple python project. Lets add the necessary files and structure to create the package, then build the package and upload it to python package index ( PyPI ).

Let suppose your project (sample_pkg) is created in the root directory.

Mandatory Basic Directory Structure of your sample project should be:

firstly you need to edit your  __init__.py  file so that later on you can verify its installed correctly.

//edit your  file with:

The most important file which is required in packaging your python project is setup.py. Your setup.py file should be in the root directory of your project.

Setup.py is the build script for setuptools. Setuptools which is a collection of Python dist-utils which allows you to more easily build and distribute Python distributions, specifically those ones that have dependencies on other packages.

//setup.py file:

you can customize your setup.py file. Important variables inside the setup function are:

  • name: name of your package, contains letters, numbers, _ and – .
  • version: should be the package version.
  • packages: list of all python import packages, instead of manually importing packages one by one we can use find_packages()  function to automatically discover all packages and sub-packages.
  • classifiers: tells the python package index and pip some additional metadata about your package, mandatory three things which you should always include at least the versions of Python your package works on, which license your package is available under and which operating system your package will work on. Complete list of classifiers is here

Firstly ensure that you have the latest version of setuptools installed :

python3 -m pip install –user –upgrade setuptools

Then, run the below command from the directory of your project where setup.py is located.

python3 setup.py sdist

// The tar.gz file is source archive and the index file is an html file having the link of the tar file.

 

Hosting The Python Repository:

For hosting the python repository firstly you should have s3pypi command line tool installed or you can install it by the command:

sudo pip install -U s3pypi

 

To publish a package:

In order to upload your package to the repository,

  1. cd to the root directory of your project.
  2. create an s3 bucket on AWS.
  3. ensure your project contains a setup script.

Run command :

 

To install a package:

Install your packages from s3 using pip by pointing the –extra-index-url as your bucket url/bucket_name with secure connection.

Pip doesn’t have config file by default so we can create our own pip.config file. 

So Alternative of installing the package from s3 by setting or pointing to the url of s3 in config file as:

 

 


references:

https://packaging.python.org/tutorials/packaging-projects/

https://novemberfive.co/blog/opensource-pypi-package-repository-tutorial/

 

Advertisements

Written by 

Neelam is a software consultant with experience of more than 6 months. She likes to keep up with the trending technologies. She is familiar with languages such as C, C++, Java, Scala, Angular 4 and is currently working on lagom with java. Her hobbies includes playing Table Tennis and listening music.

Knoldus Pune Careers - Hiring Freshers

Get a head start on your career at Knoldus. Join us!