Loading repository data…
CSU-Theory-Suite / repository
This repository holds the source files and html files of a documentation webpage made by the members of the Patonlab and BioE-KimLab at CSU
.. contents:: :depth: 2
This repo contains documentation created by members of the
@Patonlab <https://github.com/patonlab>__ and the
@BioE-KimLab <https://github.com/BioE-KimLab>__
for the Theory Suite <https://www.natsci.colostate.edu/tours-chemistry-computational-suite/>_
at Colorado State University.
The contents are focused in the area of computational chemistry and range from example tutorials of calculations to useful resources for new or existing members.
This repository fulfills three goals:
Even if you are not a member of either of the research labs, please feel free to explore and provide feedback on the contents of this repository.
As you can see, the repository contains two main folders, the :code:docs folder
and the :code:sphinx folder. The :code:docs folder contains all the relevant
files for hosting the webpage on github-pages, in almost all of the cases you
will not be editing any of those files manually so you should focus on the
:code:sphinx folder. This is the root folder of our wiki and we can find
the :code:source folder in it. This second folder contains all the files
needed for generating the final html files that will end up in the :code:docs
folder.
Here we have one folder for each one of the main sections of our webpage. Per each one of the subsections we will always find at least one .rst file. If it requires lots of additional resources or its subsequent subsections are long, we will find a folder with the same name and within this folder we will find the appropriate contents, be it a folder with all the image files, or all the subsequent subsections' .rst files.
Here is a short walkthrough for making changes to the webpage, such as correcting typos or adding tutorials. There is a slightly more detailed version of these same instructions within the wiki website.
On the top-right of the repository page you should be able to see a "Fork" button (between "Watch" and "Star"). This button allows you to create a clone of the current repository in your own GitHub account. This is now your own repository and you can make and implement changes as you see fit without editing the primary page (owned by CSU-Theory-Suite).
Rather that appearing as :code:CSU-Theory-Suite/theory-suite-wiki, your fork of this repository will be shown
as :code:YOUR-GITHUB-USERNAME/theory-suite-wiki (or something else if you changed the name).
Cloning this repository on your own computer will allow you to make edits more easily using programs such as Visual Studio Code, or even just the terminal. However, it is important to ensure that the clone on your computer is updated the same as your forked repository on the GitHub website, and possibly that your online fork is updated to the primary repository owned by CSU-Theory-Suite.
To clone your fork onto your computer, open the repository page of the fork you just created, go to the green "<> Code" button at the top right, and copy your preferred URL.
Then, open a terminal and navigate to the folder where
you will be getting a local copy of the repository. We will assume that the
:code:Documents/theory-suite-wiki is where we want our local copy.
.. code:: shell
cd Documents git clone https://github.com/CSU-Theory-Suite/theory-suite-wiki.git theory-suite-wiki
Although we do not need many special tools for building our environment, we do need to make sure that we have the basic necessary tools:
We can install them easily using pip
.. code:: shell
python -m pip install sphinx sphinx-rtd-theme nbsphinx
Warning! Make sure that the spelling is correct, the number of times I have tried to install sphinx-rdt-theme and got annoyed at it being unable to find the package have been more than what I would like to acknowledge.
Note: Using :code:python -m pip instead of directly using :code:pip
will make sure that we are installing the packages in the specific python
interpreter that we are calling with the python command. If we are in a
virtual environment it will be the python interpreter of the virtual
environment, independently of it being a conda environment, a python venv
or any other kind of virtual environment (or no virtual environment whatsoever).
We can generate a local preview of the html files (the website page we are creating) with the following commands:
.. code:: shell
cd ~/Documents/csu-theory-suite/sphinx make clean # not always required, but will make sure that all changes are reflected make html
This will create a new folder with the name :code:_build within the sphinx folder.
Within this folder we have the :code:html folder that contains the :code:index.html
A simple way of pre-visualizing the documentation is to drag and drop this file
to our web browser and navigate to the section we had edited.
Using our preferred text editor, we will make the wanted changes and save those files. For more details on how to edit the rst files, check within the wiki website. After a set of related changes we will "locally save them" in our git repository. As an example, let's assume we made changes in one file of graphical_software and another of helpful_packages.
.. code:: shell
cd ~/Documents/csu-theory-suite/sphinx git status # Check that all the files in red are the ones that you have changed git add source git status # Check that all the files now appear in green git commit -m 'Descriptive short message summarizing the changes'
Note: Remember to re-generate the local preview and visualize the changes to ensure that they show properly and double check for typos.
Now that we have changed the source files, we want to have them not only on our local computer but also on our personal GitHub. To do so we just need to run:
.. code:: shell
git push origin master
This command will sync the online version of your fork with the version you have made edits to on your computer. Similarly, if you are trying to sync the version on your computer to match that which is online, the command is:
.. code:: shell
git pull origin master
Now we proceed to create a pull request. We go to the repository page of our personal fork, click on the "Pull requests" tab (at the top left of the page) and then on the green button "New pull request" at the top right of the page.
Now we make sure that the base repository is the CSU-Theory-Suite/theory-suite-wiki (on the left) and that the head repository is ours (on the right) and click the "Create pull request"
Finally, we write a short message as title and description of the changes we made. This is likely going to be read by whoever is acting as admin of the repo before adding your changes to the primary webpage. They might provide feedback on the changes (especially if you do not ask them to merge it 5 minutes before the web needs to be active for, lets say, a presentation)
After writing the message, click on "Create pull request". Now its the admins' problem! They should be notified by GitHub, but it never hurts to send a message letting them know you created a pull request.
The current admins of this page are Alex (Paton group), Jake (Paton group), and Hojin (Kim group). Feel free to ask about becoming an admin yourself if you're interested!
Note: Even if the merge is accepted on the spot, it will still have a short delay for the changes to be reflected in the actual webpage. Usually you can keep track of it by checking a yellow/green dot that will appear in the last commit of the CSU-Theory-Suite repo. Once you see the green checkmark, everything should be good to go.
Selected from shared topics, language and repository description—not editorial ratings.
siddartha19 /
This repository holds the assignments of the Course : HTML, CSS, and Javascript for Web Developers by Johns Hopkins University
ajaybhatiya1234 /
 Read the technical deep dive: https://www.dessa.com/post/deepfake-detection-that-actually-works # Visual DeepFake Detection In our recent [article](https://www.dessa.com/post/deepfake-detection-that-actually-works), we make the following contributions: * We show that the model proposed in current state of the art in video manipulation (FaceForensics++) does not generalize to real-life videos randomly collected from Youtube. * We show the need for the detector to be constantly updated with real-world data, and propose an initial solution in hopes of solving deepfake video detection. Our Pytorch implementation, conducts extensive experiments to demonstrate that the datasets produced by Google and detailed in the FaceForensics++ paper are not sufficient for making neural networks generalize to detect real-life face manipulation techniques. It also provides a current solution for such behavior which relies on adding more data. Our Pytorch model is based on a pre-trained ResNet18 on Imagenet, that we finetune to solve the deepfake detection problem. We also conduct large scale experiments using Dessa's open source scheduler + experiment manger [Atlas](https://github.com/dessa-research/atlas). ## Setup ## Prerequisities To run the code, your system should meet the following requirements: RAM >= 32GB , GPUs >=1 ## Steps 0. Install [nvidia-docker](https://github.com/nvidia/nvidia-docker/wiki/Installation-(version-2.0)) 00. Install [ffmpeg](https://www.ffmpeg.org/download.html) or `sudo apt install ffmpeg` 1. Git Clone this repository. 2. If you haven't already, install [Atlas](https://github.com/dessa-research/atlas). 3. Once you've installed Atlas, activate your environment if you haven't already, and navigate to your project folder. That's it, You're ready to go! ## Datasets Half of the dataset used in this project is from the [FaceForensics](https://github.com/ondyari/FaceForensics/tree/master/dataset) deepfake detection dataset. . To download this data, please make sure to fill out the [google form](https://github.com/ondyari/FaceForensics/#access) to request access to the data. For the dataset that we collected from Youtube, it is accessible on [S3](ttps://deepfake-detection.s3.amazonaws.com/augment_deepfake.tar.gz) for download. To automatically download and restructure both datasets, please execute: ``` bash restructure_data.sh faceforensics_download.py ``` Note: You need to have received the download script from FaceForensics++ people before executing the restructure script. Note2: We created the `restructure_data.sh` to do a split that replicates our exact experiments avaiable in the UI above, please feel free to change the splits as you wish. ## Walkthrough Before starting to train/evaluate models, we should first create the docker image that we will be running our experiments with. To do so, we already prepared a dockerfile to do that inside `custom_docker_image`. To create the docker image, execute the following commands in terminal: ``` cd custom_docker_image nvidia-docker build . -t atlas_ff ``` Note: if you change the image name, please make sure you also modify line 16 of `job.config.yaml` to match the docker image name. Inside `job.config.yaml`, please modify the data path on host from `/media/biggie2/FaceForensics/datasets/` to the absolute path of your `datasets` folder. The folder containing your datasets should have the following structure: ``` datasets ├── augment_deepfake (2) │ ├── fake │ │ └── frames │ ├── real │ │ └── frames │ └── val │ ├── fake │ └── real ├── base_deepfake (1) │ ├── fake │ │ └── frames │ ├── real │ │ └── frames │ └── val │ ├── fake │ └── real ├── both_deepfake (3) │ ├── fake │ │ └── frames │ ├── real │ │ └── frames │ └── val │ ├── fake │ └── real ├── precomputed (4) └── T_deepfake (0) ├── manipulated_sequences │ ├── DeepFakeDetection │ ├── Deepfakes │ ├── Face2Face │ ├── FaceSwap │ └── NeuralTextures └── original_sequences ├── actors └── youtube ``` Notes: * (0) is the dataset downloaded using the FaceForensics repo scripts * (1) is a reshaped version of FaceForensics data to match the expected structure by the codebase. subfolders called `frames` contain frames collected using `ffmpeg` * (2) is the augmented dataset, collected from youtube, available on s3. * (3) is the combination of both base and augmented datasets. * (4) precomputed will be automatically created during training. It holds cashed cropped frames. Then, to run all the experiments we will show in the article to come, you can launch the script `hparams_search.py` using: ```bash python hparams_search.py ``` ## Results In the following pictures, the title for each subplot is in the form `real_prob, fake_prob | prediction | label`. #### Model trained on FaceForensics++ dataset For models trained on the paper dataset alone, we notice that the model only learns to detect the manipulation techniques mentioned in the paper and misses all the manipulations in real world data (from data)   #### Model trained on Youtube dataset Models trained on the youtube data alone learn to detect real world deepfakes, but also learn to detect easy deepfakes in the paper dataset as well. These models however fail to detect any other type of manipulation (such as NeuralTextures).   #### Model trained on Paper + Youtube dataset Finally, models trained on the combination of both datasets together, learns to detect both real world manipulation techniques as well as the other methods mentioned in FaceForensics++ paper.   for a more in depth explanation of these results, please refer to the [article](https://www.dessa.com/post/deepfake-detection-that-actually-works) we published. More results can be seen in the [interactive UI](http://deepfake-detection.dessa.com/projects) ## Help improve this technology Please feel free to fork this work and keep pushing on it. If you also want to help improving the deepfake detection datasets, please share your real/forged samples at foundations@dessa.com. ## LICENSE © 2020 Square, Inc. ATLAS, DESSA, the Dessa Logo, and others are trademarks of Square, Inc. All third party names and trademarks are properties of their respective owners and are used for identification purposes only.
Maximinodotpy /
This Repository holds the code to my Tutorials that can be read at https://maximmaeder.com
r3ap3rpy /
This repository holds a python based webapp which renders markdown pages to html.
smith-jj /
# Employee Database: A Mystery in Two Parts  ## Background It is a beautiful spring day, and it is two weeks since you have been hired as a new data engineer at Pewlett Hackard. Your first major task is a research project on employees of the corporation from the 1980s and 1990s. All that remain of the database of employees from that period are six CSV files. In this assignment, you will design the tables to hold data in the CSVs, import the CSVs into a SQL database, and answer questions about the data. In other words, you will perform: 1. Data Modeling 2. Data Engineering 3. Data Analysis ## Instructions #### Data Modeling Inspect the CSVs and sketch out an ERD of the tables. Feel free to use a tool like [http://www.quickdatabasediagrams.com](http://www.quickdatabasediagrams.com). #### Data Engineering * Use the information you have to create a table schema for each of the six CSV files. Remember to specify data types, primary keys, foreign keys, and other constraints. * Import each CSV file into the corresponding SQL table. #### Data Analysis Once you have a complete database, do the following: 1. List the following details of each employee: employee number, last name, first name, gender, and salary. 2. List employees who were hired in 1986. 3. List the manager of each department with the following information: department number, department name, the manager's employee number, last name, first name, and start and end employment dates. 4. List the department of each employee with the following information: employee number, last name, first name, and department name. 5. List all employees whose first name is "Hercules" and last names begin with "B." 6. List all employees in the Sales department, including their employee number, last name, first name, and department name. 7. List all employees in the Sales and Development departments, including their employee number, last name, first name, and department name. 8. In descending order, list the frequency count of employee last names, i.e., how many employees share each last name. ## Bonus (Optional) As you examine the data, you are overcome with a creeping suspicion that the dataset is fake. You surmise that your boss handed you spurious data in order to test the data engineering skills of a new employee. To confirm your hunch, you decide to take the following steps to generate a visualization of the data, with which you will confront your boss: 1. Import the SQL database into Pandas. (Yes, you could read the CSVs directly in Pandas, but you are, after all, trying to prove your technical mettle.) This step may require some research. Feel free to use the code below to get started. Be sure to make any necessary modifications for your username, password, host, port, and database name: ```sql from sqlalchemy import create_engine engine = create_engine('postgresql://localhost:5432/<your_db_name>') connection = engine.connect() ``` * Consult [SQLAlchemy documentation](https://docs.sqlalchemy.org/en/latest/core/engines.html#postgresql) for more information. * If using a password, do not upload your password to your GitHub repository. See [https://www.youtube.com/watch?v=2uaTPmNvH0I](https://www.youtube.com/watch?v=2uaTPmNvH0I) and [https://martin-thoma.com/configuration-files-in-python/](https://martin-thoma.com/configuration-files-in-python/) for more information. 2. Create a bar chart of average salary by title. 3. You may also include a technical report in markdown format, in which you outline the data engineering steps taken in the homework assignment. ## Epilogue Evidence in hand, you march into your boss's office and present the visualization. With a sly grin, your boss thanks you for your work. On your way out of the office, you hear the words, "Search your ID number." You look down at your badge to see that your employee ID number is 499942. ## Submission * Create an image file of your ERD. * Create a `.sql` file of your table schemata. * Create a `.sql` file of your queries. * (Optional) Create a Jupyter Notebook of the bonus analysis. * Create and upload a repository with the above files to GitHub and post a link on BootCamp Spot.
ashleemboyer /
This repository will hold the course example for the upcoming Build an Accessible Blog with Next.js and Semantic HTML course.