What is JavaScript and what does it do? Before you start learning something new, it’s important to understand exactly what it is and what it does. This is especially useful when it comes to mastering a new programming language. In simple terms, JavaScript is a programming language used to make websites interactive. If you think about the basic makeup of a website, you have HTML, which describes and defines the basic content and structure of the website, then you have CSS, which tells the browser how this HTML content should be displayed—determining things like color and font. With just HTML and CSS, you have a website that looks good but doesn’t actually do much. JavaScript brings the website to life by adding functionality. JavaScript is responsible for elements that the user can interact with, such as drop-down menus, modal windows, and contact forms. It is also used to create things like animations, video players, and interactive maps. Nowadays, JavaScript is an all-purpose programming language—meaning it runs across the entire software stack. The most popular application of JavaScript is on the client side (aka frontend), but since Node.js came on the scene, many people run JavaScript on the server side (aka backend) as well. When used on the client side, JavaScript code is read, interpreted, and executed in the user’s web browser. When used on the server side, it is run on a remote computer. You can learn more about the difference between frontend and backend programming here. JavaScript isn’t only used to create websites. It can also be used to build browser-based games and, with the help of certain frameworks, mobile apps for different operating systems. The creation of new libraries and frameworks is also making it possible to build backend programs with JavaScript, such as web apps and server apps. Is it still worth learning JavaScript in 2021? The world of web development is constantly moving. With so many new tools popping up all the time, it can be extremely difficult to know where you should focus your efforts. As an aspiring developer, you’ll want to make sure that what you’re learning is still relevant in today’s industry. If you’re having doubts about JavaScript, it’s important to know that, since its creation in 1995, JavaScript is pretty much everywhere on the web—and that’s not likely to change any time soon. According to the 2020 StackOverflow developer survey, JavaScript is the most commonly used programming language for the eighth year in a row. It is currently used by 94.5% of all websites and, despite originally being designed as a client-side language, JavaScript has now made its way to the server-side of websites (thanks to Node.js), mobile devices (thanks to React Native and Ionic) and desktop (courtesy of Electron). As long as people are interacting with the web, you can assume that JavaScript is highly relevant—there’s no doubt that this is a language worth knowing! With that in mind, let’s look at some of the key benefits of becoming a JavaScript expert. Why learn JavaScript? The most obvious reason for learning JavaScript is if you have hopes of becoming a web developer. Even if you haven’t got your heart set on a tech career, being proficient in JavaScript will enable you to build websites from scratch—a pretty useful skill to have in today’s job market! If you do want to become a web developer, here are some of the main reasons why you should learn JavaScript: JavaScript experts are versatile JavaScript is an extremely versatile language. Once you’ve mastered it, the possibilities are endless: you can code on the client-side (frontend) using Angular and on the server-side (backend) using Node.js. You can also develop web, mobile, and desktop apps using React, React Native, and Electron, and you can even get involved in machine learning. If you want to become a frontend developer, JavaScript is a prerequisite. However, that’s not the only career path open to you as a JavaScript expert. Mastering this key programming language could see you go on to work in full-stack development, games development, information security software engineering, machine learning, and artificial intelligence—to name just a few! Ultimately, if you want any kind of development or engineering career, proficiency in JavaScript is a must. JavaScript experts are in-demand (and well-paid) JavaScript is the most popular programming language in the world, so it’s no wonder that JavaScript is one of the most sought-after skills in the web development industry today. According to the Devskiller IT Skills and Hiring Report 2020, 72% of companies are looking to hire JavaScript experts. Enter the search term “JavaScript” on job site Indeed and you’ll find over 40,000 jobs requiring this skill (in the US). Run the same search on LinkedIn and the results are in excess of 125,000. At the same time, the global demand for JavaScript seems to outweigh the expertise available on the market. According to this 2018 HackerRank report, 48% of employers worldwide need developers with JavaScript skills, while only 42% of student developers claim to be proficient in JavaScript. And, in their most recent report for 2020, HackerRank once again reports that JavaScript is the most popular language that hiring mangers look for in a web developer candidate. Not only are JavaScript experts in demand—they are also well-paid. In the United States, JavaScript developers earn an average yearly salary of $111,953 per year. We’ve covered this topic in more detail in our JavaScript salary guide, but as you can see, learning JavaScript can really boost your earning potential as a developer. JavaScript is beginner-friendly Compared to many other programming languages, JavaScript offers one of the more beginner-friendly entry points into the world of coding. The great thing about JavaScript is that it comes installed on every modern web browser—there’s no need to set up any kind of development environment, which means you can start coding with JavaScript right away! Another advantage of learning JavaScript as your first programming language is that you get instant feedback; with a minimal amount of JavaScript code, you’ll immediately see visible results. There’s also a huge JavaScript community on sites like Stack Overflow, so you’ll find plenty of support as you learn. Not only is JavaScript beginner-friendly; it will also set you up with some extremely valuable transferable skills. JavaScript supports object-oriented, functional, and imperative styles of programming—skills which can be transferred to any new language you might learn later on, such as Python, Java, or C++. JavaScript provides a crucial introduction to key principles and practices that you’ll take with you throughout your career as a developer. Should you learn plain JavaScript first or can you skip to frameworks and libraries? When deciding whether or not to learn JavaScript, what you’re really asking is whether or not you should learn “vanilla” JavaScript. Vanilla JavaScript just means plain JavaScript without any libraries or frameworks. Let’s explore what this means in more detail now. What is meant by vanilla JavaScript, libraries, and frameworks? If you research the term “vanilla JavaScript”, you might run into some confusion; however, all you need to know is that vanilla JavaScript is used to refer to native, standards-based, non-extended JavaScript. There is no difference between vanilla JavaScript and JavaScript—it’s just there to emphasize the usage of plain JavaScript without the use of libraries and frameworks. So what are libraries and frameworks? JavaScript libraries and frameworks both contain sets of prewritten, ready-to-use JavaScript code—but they’re not the same thing. You can think of a framework as your blueprint for building a website: it gives you a structure to work from, and contains ready-made components and tools that help you to build certain elements much quicker than if you were to code them from scratch. Some popular JavaScript frameworks include Angular, React, Vue, and Node.js. Frameworks also contain libraries. Libraries are smaller than frameworks, and tend to be used for more specific cases. A JavaScript library contains sets of JavaScript code which can be called upon to implement certain functions and features. Let’s imagine you want to code a particular element into your website. You could write, say, ten lines of JavaScript from scratch—or you could take the condensed, ready-made version from your chosen JavaScript library. Some examples of JavaScript libraries include jQuery, Lodash, and Underscore. The easiest way to understand how frameworks and libraries work together is to imagine you are building a house. The framework provides the foundation and the structure, while the library enables you to add in ready-made components (like furniture) rather than building your own from scratch. You can learn more about the relationship between languages and libraries in this post explaining the main differences between JavaScript and jQuery. For now, let’s go back to our original question: How important is it to learn vanilla JavaScript? Should you learn vanilla JavaScript first? When it comes to learning JavaScript, it can be tempting to skip ahead to those time-saving frameworks and libraries we just talked about—and many developers do. However, there are many compelling arguments for learning plain JavaScript first. While JavaScript frameworks may help you get the job done quicker, there’s only so far you can go if you don’t understand the core concepts behind these frameworks. Frontend developer Abhishek Nagekar describes how not learning vanilla JavaScript came back to bite him when he started learning the JavaScript frameworks Node and Express: “As I went to write more and more code in Node and Express, I began to get stuck at even the tiniest problems. Suddenly, I was surrounded with words like callbacks, closures, event loop and prototype. It felt like I got a reintroduction to JavaScript, but this time, it was not a toddler playing in its cradle, it was something of a mysterious monster, challenging me on every other step for not having taken it seriously.” The above Tweet references a long-running joke within the developer community, and although it dates way back to 2015, it’s still highly relevant today. If you want to become a developer who can innovate, not just execute, you need to understand the underlying principles of the web—not just the shortcuts. This means learning vanilla JavaScript before you move on to frameworks. In fact, understanding plain JavaScript will help you later on when it comes to deciding whether to use a framework for a certain project, and if so, which framework to use. Why Study JavaScript? JavaScript is one of the 3 languages all web developers must learn: 1. HTML to define the content of web pages 2. CSS to specify the layout of web pages 3. JavaScript to program the behavior of web pages Learning Speed In this tutorial, the learning speed is your choice. Everything is up to you. If you are struggling, take a break, or re-read the material. Always make sure you understand all the "Try-it-Yourself" examples. The only way to become a clever programmer is to: Practice. Practice. Practice. Code. Code. Code ! Commonly Asked Questions How do I get JavaScript? Where can I download JavaScript? Is JavaScript Free? You don't have to get or download JavaScript. JavaScript is already running in your browser on your computer, on your tablet, and on your smart-phone. JavaScript is free to use for everyone.

{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Guide to Web Scraping\n", "\n", "Let's get you started with web scraping and Python. Before we begin, here are some important rules to follow and understand:\n", "\n", "1. Always be respectful and try to get premission to scrape, do not bombard a website with scraping requests, otherwise your IP address may be blocked!\n", "2. Be aware that websites change often, meaning your code could go from working to totally broken from one day to the next.\n", "3. Pretty much every web scraping project of interest is a unique and custom job, so try your best to generalize the skills learned here.\n", "\n", "OK, let's get started with the basics!\n", "\n", "## Basic components of a WebSite\n", "\n", "### HTML\n", "HTML stands for Hypertext Markup Language and every website on the internet uses it to display information. Even the jupyter notebook system uses it to display this information in your browser. If you right click on a website and select \"View Page Source\" you can see the raw HTML of a web page. This is the information that Python will be looking at to grab information from. Let's take a look at a simple webpage's HTML:\n", "\n", " <!DOCTYPE html> \n", " <html> \n", " <head>\n", " <title>Title on Browser Tab</title>\n", " </head>\n", " <body>\n", " <h1> Website Header </h1>\n", " <p> Some Paragraph </p>\n", " <body>\n", " </html>" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's breakdown these components.\n", "\n", "Every <tag> indicates a specific block type on the webpage:\n", "\n", " 1.<DOCTYPE html> HTML documents will always start with this type declaration, letting the browser know its an HTML file.\n", " 2. The component blocks of the HTML document are placed between <html> and </html>.\n", " 3. Meta data and script connections (like a link to a CSS file or a JS file) are often placed in the <head> block.\n", " 4. The <title> tag block defines the title of the webpage (its what shows up in the tab of a website you're visiting).\n", " 5. Is between <body> and </body> tags are the blocks that will be visible to the site visitor.\n", " 6. Headings are defined by the <h1> through <h6> tags, where the number represents the size of the heading.\n", " 7. Paragraphs are defined by the <p> tag, this is essentially just normal text on the website.\n", "\n", " There are many more tags than just these, such as <a> for hyperlinks, <table> for tables, <tr> for table rows, and <td> for table columns, and more!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### CSS\n", "\n", "CSS stands for Cascading Style Sheets, this is what gives \"style\" to a website, including colors and fonts, and even some animations! CSS uses tags such as **id** or **class** to connect an HTML element to a CSS feature, such as a particular color. **id** is a unique id for an HTML tag and must be unique within the HTML document, basically a single use connection. **class** defines a general style that can then be linked to multiple HTML tags. Basically if you only want a single html tag to be red, you would use an id tag, if you wanted several HTML tags/blocks to be red, you would create a class in your CSS doc and then link it to the rest of these blocks." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Scraping Guidelines\n", "\n", "Keep in mind you should always have permission for the website you are scraping! Check a websites terms and conditions for more info. Also keep in mind that a computer can send requests to a website very fast, so a website may block your computer's ip address if you send too many requests too quickly. Lastly, websites change all the time! You will most likely need to update your code often for long term web-scraping jobs." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Web Scraping with Python\n", "\n", "There are a few libraries you will need, you can go to your command line and install them with conda install (if you are using anaconda distribution), or pip install for other python distributions.\n", "\n", " conda install requests\n", " conda install lxml\n", " conda install bs4\n", " \n", "if you are not using the Anaconda Installation, you can use **pip install** instead of **conda install**, for example:\n", "\n", " pip install requests\n", " pip install lxml\n", " pip install bs4\n", " \n", "Now let's see what we can do with these libraries.\n", "\n", "----" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example Task 0 - Grabbing the title of a page\n", "\n", "Let's start very simple, we will grab the title of a page. Remember that this is the HTML block with the **title** tag. For this task we will use **www.example.com** which is a website specifically made to serve as an example domain. Let's go through the main steps:" ] }, { "cell_type": "code", "execution_count": 51, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import requests" ] }, { "cell_type": "code", "execution_count": 52, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Step 1: Use the requests library to grab the page\n", "# Note, this may fail if you have a firewall blocking Python/Jupyter \n", "# Note sometimes you need to run this twice if it fails the first time\n", "res = requests.get(\"http://www.example.com\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This object is a requests.models.Response object and it actually contains the information from the website, for example:" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "requests.models.Response" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(res)" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'<!doctype html>\\n<html>\\n<head>\\n <title>Example Domain</title>\\n\\n <meta charset=\"utf-8\" />\\n <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\\n <style type=\"text/css\">\\n body {\\n background-color: #f0f0f2;\\n margin: 0;\\n padding: 0;\\n font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\\n \\n }\\n div {\\n width: 600px;\\n margin: 5em auto;\\n padding: 2em;\\n background-color: #fdfdff;\\n border-radius: 0.5em;\\n box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\\n }\\n a:link, a:visited {\\n color: #38488f;\\n text-decoration: none;\\n }\\n @media (max-width: 700px) {\\n div {\\n margin: 0 auto;\\n width: auto;\\n }\\n }\\n </style> \\n</head>\\n\\n<body>\\n<div>\\n <h1>Example Domain</h1>\\n <p>This domain is for use in illustrative examples in documents. You may use this\\n domain in literature without prior coordination or asking for permission.</p>\\n <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\\n</div>\\n</body>\\n</html>\\n'" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res.text" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "____\n", "Now we use BeautifulSoup to analyze the extracted page. Technically we could use our own custom script to loook for items in the string of **res.text** but the BeautifulSoup library already has lots of built-in tools and methods to grab information from a string of this nature (basically an HTML file). Using BeautifulSoup we can create a \"soup\" object that contains all the \"ingredients\" of the webpage. Don't ask me about the weird library names, I didn't choose them! :)" ] }, { "cell_type": "code", "execution_count": 55, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import bs4" ] }, { "cell_type": "code", "execution_count": 56, "metadata": { "collapsed": true }, "outputs": [], "source": [ "soup = bs4.BeautifulSoup(res.text,\"lxml\")" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "<!DOCTYPE html>\n", "<html>\n", "<head>\n", "<title>Example Domain</title>\n", "<meta charset=\"utf-8\"/>\n", "<meta content=\"text/html; charset=utf-8\" http-equiv=\"Content-type\"/>\n", "<meta content=\"width=device-width, initial-scale=1\" name=\"viewport\"/>\n", "<style type=\"text/css\">\n", " body {\n", " background-color: #f0f0f2;\n", " margin: 0;\n", " padding: 0;\n", " font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n", " \n", " }\n", " div {\n", " width: 600px;\n", " margin: 5em auto;\n", " padding: 2em;\n", " background-color: #fdfdff;\n", " border-radius: 0.5em;\n", " box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n", " }\n", " a:link, a:visited {\n", " color: #38488f;\n", " text-decoration: none;\n", " }\n", " @media (max-width: 700px) {\n", " div {\n", " margin: 0 auto;\n", " width: auto;\n", " }\n", " }\n", " </style>\n", "</head>\n", "<body>\n", "<div>\n", "<h1>Example Domain</h1>\n", "<p>This domain is for use in illustrative examples in documents. You may use this\n", " domain in literature without prior coordination or asking for permission.</p>\n", "<p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n", "</div>\n", "</body>\n", "</html>" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" } ], "source": [ "soup" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let's use the **.select()** method to grab elements. We are looking for the 'title' tag, so we will pass in 'title'\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[<title>Example Domain</title>]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "soup.select('title')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice what is returned here, its actually a list containing all the title elements (along with their tags). You can use indexing or even looping to grab the elements from the list. Since this object it still a specialized tag, we cna use method calls to grab just the text." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "title_tag = soup.select('title')" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "<title>Example Domain</title>" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "title_tag[0]" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bs4.element.Tag" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(title_tag[0])" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Example Domain'" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "title_tag[0].getText()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example Task 1 - Grabbing all elements of a class\n", "\n", "Let's try to grab all the section headings of the Wikipedia Article on Grace Hopper from this URL: https://en.wikipedia.org/wiki/Grace_Hopper" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# First get the request\n", "res = requests.get('https://en.wikipedia.org/wiki/Grace_Hopper')" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Create a soup from request\n", "soup = bs4.BeautifulSoup(res.text,\"lxml\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now its time to figure out what we are actually looking for. Inspect the element on the page to see that the section headers have the class \"mw-headline\". Because this is a class and not a straight tag, we need to adhere to some syntax for CSS. In this case" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<table>\n", "\n", "<thead >\n", "<tr>\n", "<th>\n", "<p>Syntax to pass to the .select() method</p>\n", "</th>\n", "<th>\n", "<p>Match Results</p>\n", "</th>\n", "</tr>\n", "</thead>\n", "<tbody>\n", "<tr>\n", "<td>\n", "<p><code>soup.select('div')</code></p>\n", "</td>\n", "<td>\n", "<p>All elements with the <code>&lt;div&gt;</code> tag</p>\n", "</td>\n", "</tr>\n", "<tr>\n", "<td>\n", "<p><code>soup.select('#some_id')</code></p>\n", "</td>\n", "<td>\n", "<p>The HTML element containing the <code>id</code> attribute of <code>some_id</code></p>\n", "</td>\n", "</tr>\n", "<tr>\n", "<td>\n", "<p><code>soup.select('.notice')</code></p>\n", "</td>\n", "<td>\n", "<p>All the HTML elements with the CSS <code>class</code> named <code>notice</code></p>\n", "</td>\n", "</tr>\n", "<tr>\n", "<td>\n", "<p><code>soup.select('div span')</code></p>\n", "</td>\n", "<td>\n", "<p>Any elements named <code>&lt;span&gt;</code> that are within an element named <code>&lt;div&gt;</code></p>\n", "</td>\n", "</tr>\n", "<tr>\n", "<td>\n", "<p><code>soup.select('div &gt; span')</code></p>\n", "</td>\n", "<td>\n", "<p>Any elements named <code class=\"literal2\">&lt;span&gt;</code> that are <span><em >directly</em></span> within an element named <code class=\"literal2\">&lt;div&gt;</code>, with no other element in between</p>\n", "</td>\n", "</tr>\n", "<tr>\n", "\n", "</tr>\n", "</tbody>\n", "</table>" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[<span class=\"mw-headline\" id=\"Early_life_and_education\">Early life and education</span>,\n", " <span class=\"mw-headline\" id=\"Career\">Career</span>,\n", " <span class=\"mw-headline\" id=\"World_War_II\">World War II</span>,\n", " <span class=\"mw-headline\" id=\"UNIVAC\">UNIVAC</span>,\n", " <span class=\"mw-headline\" id=\"COBOL\">COBOL</span>,\n", " <span class=\"mw-headline\" id=\"Standards\">Standards</span>,\n", " <span class=\"mw-headline\" id=\"Retirement\">Retirement</span>,\n", " <span class=\"mw-headline\" id=\"Post-retirement\">Post-retirement</span>,\n", " <span class=\"mw-headline\" id=\"Anecdotes\">Anecdotes</span>,\n", " <span class=\"mw-headline\" id=\"Death\">Death</span>,\n", " <span class=\"mw-headline\" id=\"Dates_of_rank\">Dates of rank</span>,\n", " <span class=\"mw-headline\" id=\"Awards_and_honors\">Awards and honors</span>,\n", " <span class=\"mw-headline\" id=\"Military_awards\">Military awards</span>,\n", " <span class=\"mw-headline\" id=\"Other_awards\">Other awards</span>,\n", " <span class=\"mw-headline\" id=\"Legacy\">Legacy</span>,\n", " <span class=\"mw-headline\" id=\"Places\">Places</span>,\n", " <span class=\"mw-headline\" id=\"Programs\">Programs</span>,\n", " <span class=\"mw-headline\" id=\"In_popular_culture\">In popular culture</span>,\n", " <span class=\"mw-headline\" id=\"Grace_Hopper_Celebration_of_Women_in_Computing\">Grace Hopper Celebration of Women in Computing</span>,\n", " <span class=\"mw-headline\" id=\"Notes\">Notes</span>,\n", " <span class=\"mw-headline\" id=\"Obituary_notices\">Obituary notices</span>,\n", " <span class=\"mw-headline\" id=\"See_also\">See also</span>,\n", " <span class=\"mw-headline\" id=\"References\">References</span>,\n", " <span class=\"mw-headline\" id=\"Further_reading\">Further reading</span>,\n", " <span class=\"mw-headline\" id=\"External_links\">External links</span>]" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# note depending on your IP Address, \n", "# this class may be called something different\n", "soup.select(\".toctext\")" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Early life and education\n", "Career\n", "World War II\n", "UNIVAC\n", "COBOL\n", "Standards\n", "Retirement\n", "Post-retirement\n", "Anecdotes\n", "Death\n", "Dates of rank\n", "Awards and honors\n", "Military awards\n", "Other awards\n", "Legacy\n", "Places\n", "Programs\n", "In popular culture\n", "Grace Hopper Celebration of Women in Computing\n", "Notes\n", "Obituary notices\n", "See also\n", "References\n", "Further reading\n", "External links\n" ] } ], "source": [ "for item in soup.select(\".toctext\"):\n", " print(item.text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example Task 3 - Getting an Image from a Website\n", "\n", "Let's attempt to grab the image of the Deep Blue Computer from this wikipedia article: https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": true }, "outputs": [], "source": [ "res = requests.get(\"https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)\")" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": true }, "outputs": [], "source": [ "soup = bs4.BeautifulSoup(res.text,'lxml')" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": true }, "outputs": [], "source": [ "image_info = soup.select('.thumbimage')" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[<img alt=\"\" class=\"thumbimage\" data-file-height=\"601\" data-file-width=\"400\" decoding=\"async\" height=\"331\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/b/be/Deep_Blue.jpg/220px-Deep_Blue.jpg\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/b/be/Deep_Blue.jpg/330px-Deep_Blue.jpg 1.5x, //upload.wikimedia.org/wikipedia/commons/b/be/Deep_Blue.jpg 2x\" width=\"220\"/>,\n", " <img alt=\"\" class=\"thumbimage\" data-file-height=\"600\" data-file-width=\"800\" decoding=\"async\" height=\"165\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/6/6f/Kasparov_Magath_1985_Hamburg-2.png/220px-Kasparov_Magath_1985_Hamburg-2.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/6/6f/Kasparov_Magath_1985_Hamburg-2.png/330px-Kasparov_Magath_1985_Hamburg-2.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/6/6f/Kasparov_Magath_1985_Hamburg-2.png/440px-Kasparov_Magath_1985_Hamburg-2.png 2x\" width=\"220\"/>]" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "image_info" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(image_info)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": true }, "outputs": [], "source": [ "computer = image_info[0]" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bs4.element.Tag" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ],

# Trybe This repository contains all the learning activities developed by João V.S Dias https://www.linkedin.com/in/diasvictorj/_ while studying at [Trybe](https://www.betrybe.com/) :rocket: _"Trybe is a school of the future for anyone who wants to improve their life and build a successful career in technology, where people only pay when they get a good job."_ The program has more than 1,500 hours of classroom and online classes, covers an introduction to software development, front-end, back-end, computer science, software engineering, agile methodologies and behavioral skills. ## Web Development Fundamentals :white_check_mark: ##### Block 1: Introduction - Unix & Shell - [ ] 1-3: _Unix & Shell- Part 1_ - [ ] 1-4: _Unix & Shell- Part 2_ ##### Block 2: Git & GitHub - [ ] 2-1: _What is it and what is it for?_ - [ ] 2-2: _Understanding the commands_ - [ ] 2-3: _Internet - Understanding how it works_ ##### Block 3: Introduction - HTML & CSS - [ ] 3-1: _HTML & CSS - Page Structures_ - [ ] 3-2: _HTML & CSS - Getting Started with CSS_ - [ ] 3-3: _HTML & CSS - Selectors and positioning_ - [ ] 3-4: _Semantic HTML_ - [ ] 3-5: _[Project - HTML & CSS]()_ ##### Block 4: Introduction - JavaScript - [ ] 4-1: _JavaScript - First steps_ - [ ] 4-2: _JavaScript - Array and For_ loop - [ ] 4-3: _JavaScript - Programming Logic and Algorithms_ - [ ] 4-4: _JavaScript - Objects and Functions_ - [ ] 4-5: _[Project - Playground Functions]()_ ##### Block 5: Introduction - JavaScript - Projects - [ ] 5-1: _JavaScript - DOM and selectors_ - [ ] 5-2: _JavaScript - Working with elements_ - [ ] 5-3: _JavaScript - Events_ - [ ] 5-4: _JavaScript - Web Storage_ - [ ] 5-5: _[Project - Meme Generator]()_ - [ ] 5-6: _[Project - Pixel Art]()_ - [ ] 5-7: _[Project - Task List]()_ - [ ] 5-7: _[Project - Guess the Color]()_ - [ ] 5-7: _[Project - Mysterious Letter]()_ ##### Block 6: Advanced HTML & CSS - [ ] 6-1: _HTML & CSS - Forms_ - [ ] 6-2: _JavaScript Libraries and CSS Frameworks_ - [ ] 6-3: _CSS Flexbox - Part 1_ - [ ] 6-4: _CSS Flexbox - Part 2_ - [ ] 6-5: _CSS Responsive - Mobile First_ - [ ] 6-6: _[Project - Facebook homepage]()_ ##### Block 7: JavaScript ES6 & Unit Tests - [ ] 7-1: _JavaScript ES6 - let, const, arrow functions and template literals_ - [ ] 7-2: _JavaScript ES6 - Objects_ - [ ] 7-3: _Unit tests in JavaScript_ - [ ] 7-4: _[Project - JavaScript Unit Tests]()_ ##### Block 8: JavaScript ES6 - [ ] 8-1: _JavaScript ES6 - Higher Order Functions - forEach, find, some, every, sort_ - [ ] 8-2: _JavaScript ES6 - Higher Order Functions - map and filter_ - [ ] 8-3: _JavaScript ES6 - Higher Order Functions - reduce_ - [ ] 8-4: _JavaScript ES6 - spread operator, rest parameter, destructuring and more_ - [ ] 8-5: _[Project - Zoo functions]()_ ##### Block 9: Asynchronicity & Callbacks - [ ] 9-1: _Asynchronous JavaScript and Callbacks_ - [ ] 9-2: _JavaScript Promises_ - [ ] 9-3: _[Project - Shopping Cart]()_ ##### Block 10: Jest - [ ] 10-1: _First steps in Jest_ - [ ] 10-2: _Jest - Asynchronous Tests_ - [ ] 10-3: _Jest - Simulating behavior_ - [ ] 10-4: _[Project - Asynchronous Jest and Mocking]()_ ## Front-end development :hourglass_flowing_sand: ##### Block 11: Introduction - React - [ ] 11-1: _'Hello, world!' on React!_ - [ ] 11-2: _React Components_ - [ ] 11-3: _[Project - Movie Cards Library]()_ ##### Block 12: React - [ ] 12-1: _Components with status_ - [ ] 12-2: _Events and forms in React_ - [ ] 12-3: _[Project - Movie Cards Library Stateful]()_ ##### Block 13: React - [ ] 13-1: _Improving component reuse: props.children and PropTypes_ - [ ] 13-2: _Component lifecycle in React_ - [ ] 13-3: _React Router_ - [ ] 13-4: _[Project - Movie Cards Library CRUD]()_ ##### Block 14: Agile Methodologies - [ ] 14-1: _Agile Methodologies_ - [ ] 14-2: _[Project - Frontend Online Store]()_ ##### Block 15: Tests in React - [ ] 15-1: _Testing React with the React Testing Library_ - [ ] 15-2: _Testing React with the React Testing Library - Part 2_ - [ ] 15-3: _[Project - Tests in React]()_ ##### Block 16: Introduction to Redux - [ ] 16-1: _Introduction to Redux_ - [ ] 16-2: _React with Redux - Part 1_ - [ ] 16-3: _React with Redux - Practice_ - [ ] 16-4: _React with Redux - Part 2_ - [ ] 16-5: _Synchronous tests with React-Redux_ - [ ] 16-6: _[Project - Table with data filters]()_ ##### Block 17: Project React - [ ] 17-1: _[Project - Trivia Game]()_ ##### Block 18: React & Context API - [ ] 18-1: _Context API of React_ - [ ] 18-2: _React Hooks - useState and useContext_ - [ ] 18-3: _React Hooks - useEffect and Custom Hooks_ - [ ] 18-4: _[Project - StarWars Datatable with Context API and Hooks]()_ ##### Block 19: Front-end Final Project - [ ] 19-1: _[Project - Recipe App]()_ ## Back-end Development :hourglass_flowing_sand: ##### Block 20: Introduction - Relational Databases - [ ] 20-1: _SQL Database_ - [ ] 20-2: _Finding data in a database_ - [ ] 20-3: _Filtering data specifically_ - [ ] 20-4: _Manipulating tables_ - [ ] 20-5: _[Project - All For One]()_ ##### Block 21: Relational Databases - [ ] 21-1: _Most used functions in SQL_ - [ ] 21-2: _Uncomplicating JOINs and UNIONs_ - [ ] 21-3: _Stored Routines & Subqueries_ - [ ] 21-4: _[Project - Vocabulary Booster]()_ ##### Block 22: Relational Databases - [ ] 22-1: _Transforming ideas into a database model_ - [ ] 22-2: _Normalization, Normal Forms and Dumps_ - [ ] 22-2: _Turning ideas into a database model - Part 2_ - [ ] 22-3: _[Project - One For All]()_ ##### Block 23: Introduction - NoSQL - [ ] 23-1: _MongoDB - Introduction_ - [ ] 23-2: _Filter Operators_ - [ ] 23-3: _[Project - Date Flights]()_ ##### Block 24: Updates - [ ] 24-1: _Simple Updates_ - [ ] 24-2: _Complex Updates - Arrays - Part 1_ - [ ] 24-3: _Complex Updates - Arrays - Part 2_ - [ ] 24-4: _[Project - Commerce]()_ ##### Block 25: Aggregation Framework - [ ] 25-1: _Aggregation Framework - Part 1_ - [ ] 25-2: _Aggregation Framework - Part 2_ - [ ] 25-3: _[Project - Aggregations]()_ ##### Block 26: Intro - NodeJS - [ ] 26-1: _NodeJS - Introduction_ - [ ] 26-2: _NodeJS - Asynchronous Flow_ - [ ] 26-3: _NodeJS - Architecture_ - [ ] 26-4: _[Project - A CLI of Ice and Fire]()_ ##### Block 27: NodeJS - [ ] 27-1: _Express: HTTP with Node.js_ - [ ] 27-2: _Software Architecture - Introduction to MVC_ - [ ] 27-3: _[Project - Cookmaster]()_ ##### Block 28: NodeJS - [ ] 28-1: _Software Architecture - Service Layer_ - [ ] 28-2: _Web Architecture - Rest and Restful_ - [ ] 28-3: _[Project - Store Manager]()_ ##### Block 29: NodeJS - [ ] 29-1: _NodeJS - JWT - (JSON Web Token)_ - [ ] 29-2: _NodeJS - Upload files with Multer_ - [ ] 29-3: _[Project - Cookmaster V2]()_ ##### Block 30: Introduction - Deploy - [ ] 30-1: _Infrastructure - Deploy with Heroku_ - [ ] 30-2: _Deploy - Process Managers_ - [ ] 30-3: _[Project - Stranger Things]()_ ##### Block 31: Project - [ ] 31-1: _[Project - Trybeer]()_ ##### Block 32: Software Architecture - [ ] 32-1: _Architecture - SOLID Principles_ - [ ] 32-2: _ORM - Application interface with the database_ - [ ] 32-3: _Software Architecture - DDD_ - [ ] 32-4: _Good practice writing tests_ - [ ] 32-3: _[Project - Blogs API]()_ ##### Block 33: Sockets - [ ] 30-1: _Sockets - TCP/UDP & NET_ - [ ] 30-2: _Sockets - Socket.io_ - [ ] 30-3: _[Project - Webchat]()_ ##### Block 34: Project - [ ] 34-1: _[Project - Trybeer V2]()_ ## Computer Science :hourglass_flowing_sand: ##### Block 35: Introduction - Computer Science - [ ] 35-1: _Computer Architecture_ - [ ] 35-2: _Network architecture_ - [ ] 35-3: _Computer networks, tools and security_ - [ ] 35-4: _[Project - Exploring the protocols]()_ ##### Block 36: Python - [ ] 35-1: _Learning Python_ - [ ] 35-2: _Tests and Exceptions_ - [ ] 35-3: _Data Input and Output_ - [ ] 36-4: _Data Input and Output_ - [ ] 35-5: _[Project - Tech news]()_ # [...]

A little Python library for making simple Electron-like HTML/JS GUI apps