spotify /
luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Loading repository data…
nirmeshk / repository
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Before we use our build management tools, we need to set up a server to handle the Github webhooks and and make sure that all the required dependencies are installed on the server. For this milestone, we elected to use Jenkins CI server to manage the builds after any repository changes. To provision a server on Digitalocean, navigate to the setup folder to run provisionServers.js and then run buildInventory.js to generate a file with the public IP address of the server. You can then ssh into your new droplet on a machine with an authorized key pair. Once you are in the server, simply run the installJenkins.sh file to initialize your build management environment
#!/bin/bash
echo ${GIT_BRANCH#*/}
rm -rf ../virtual_env/
virtualenv ../virtual_env
source ../virtual_env/bin/activate
python setup.py install
pip install tornado tox
tox -e py27-nonhdfs
Selected from shared topics, language and repository description—not editorial ratings.
spotify /
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
shirosaidev /
Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.
kirillseva /
Ruigi is a pipeline specialist, much like his python counterpart, Luigi.
th33k /
LUIGI is an interactive pet robot designed for fun, companionship, and entertainment! Featuring touchscreen controls, voice interaction, gesture recognition, remote driving, and emotional expressions, LUIGI is more than just a toy - it's a smart digital friend!
sergii1989 /
FastML Framework is a python library that allows to build effective Machine Learning solutions using luigi pipelines.
jkeung /
Analysis of movies using data webscraped from boxofficemojo.com. Scraping is parallelized using Spotify's Luigi Python package.