This is a foundational course in open-source intelligence (OSINT) gathering and, as such, will move quickly through many areas of the field. While the course is an entry point for people wanting to learn about OSINT, the concepts and tools taught are far from basic. The goal is to provide the OSINT groundwork knowledge for students to be successful in their fields, whether they are cyber defenders, threat intelligence analysts, private investigators, insurance claims investigators, intelligence analysts, law enforcement personnel, or just someone curious about OSINT.
Many people think using their favorite Internet search engine is enough to find the data they need and do not realize that most of the Internet is not indexed by search engines. SEC487 teaches students effective methods of finding these data. You will learn real-world skills and techniques that law enforcement, private investigators, cyber attackers, and defenders use to scour the massive amounts of information found on the Internet. Once you have the information, we'll show you how to ensure that it is corroborated, how to analyze what you've gathered, and how to make sure it is useful in your investigations.
You will learn OSINT by completing more than 20 hands-on exercises using the live Internet and dark web.
Additional Information:
!! IMPORTANT - BRING YOUR OWN LAPTOP CONFIGURED USING THESE DIRECTIONS!!
A properly configured system is required for each student participating in this course. Before coming to class, carefully read and follow these instructions exactly.
You can use any 64-bit version of Windows, MacOS, or Linux as your core operating system that also can install and run VMware virtualization products. You also must have 8 GB of RAM or higher for the VM to function properly in the class.
It is critical that your CPU and operating system support 64-bit so that our 64-bit guest virtual machine will run on your laptop.
In addition to having 64-bit capable hardware, AMD-V, Intel VT-x, or the equivalent must be enabled in BIOS/UEFI.
Please download and install VMware Workstation 14, VMware Fusion 10, or VMware Workstation Player 14 or higher versions on your system prior to the start of the class. If you do not own a licensed copy of VMware Workstation or Fusion, you can download a free 30-day trial copy from VMware. VMware will send you a time-limited serial number if you register for the trial on its website.
MANDATORY SEC487 SYSTEM REQUIREMENTS:
CPU: 64-bit 2.0+ GHz processor or higher-based system is mandatory for this course (Important - Please Read: a 64-bit system processor is mandatory)
BIOS/UEFI: VT-x, AMD-V, or the equivalent must be enabled in the BIOS/UEFI
RAM: 8 GB (gigabytes) of RAM or higher is mandatory for this course (Important - Please Read: 8 GB of RAM or higher is mandatory)
Wireless Ethernet 802.11 G/N/AC
USB 3.0 port (courseware provided via USB)
Disk: 30 gigabytes of free disk space
VMware Workstation 14, Workstation Player 14, or Fusion 10 (or newer)
Privileged access to the host operating system with the ability to disable security tools
A Linux virtual machine will be provided in class
Course objectives
You Will Be Able To:
Create an OSINT process
Conduct OSINT investigations in support of a wide range of customers
Understand the data collection life cycle
Create a secure platform for data collection
Analyze customer collection requirements
Capture and record data
Create sock puppet accounts
Create your own OSINT process
Harvest web data
Perform searches for people
Access social media data
Assess a remote location using online cameras and maps
Examine geolocated social media
Research businesses
Use government-provided data
Collect data from the dark web
Leverage international sites and tools
Target Audience:
This course will teach you techniques to help your work whether you are trying to find suspects for a legal investigation, identify candidates to fill a job position, gather hosts for a penetration test, or search for honey tokens as a defender.
While this list is far from complete, the OSINT topics in SEC487 will be helpful to:
Cyber Incident Responders
Digital Forensics (DFIR) analysts
Penetration Testers
Social Engineers
Law Enforcement
Intelligence Personnel
Recruiters/Sources
Private Investigators
Insurance Investigators
Human Resources Personnel
Researchers
Hands-On Labs:
SEC487 is a learn it-do it course where we examine a topic and then dive into a hands-on lab to reinforce the learning. The course has more than 20 labs spaced across the first five sections, followed by the final hands-on Capture-the-Flag challenge in section six. Check out the lab content below to get a feel for what you will be doing within our course virtual machines.
Section 1
Set up the course virtual machine and configure the VPN that is used to secure all web traffic
Use a MindMap tool to document OSINT data and then analyze relationships between people using a data visualization application
Set up a password manager to securely store all the passwords that we will need for our sock puppets and other accounts
Create a sock puppet account with realistic user-attributes, which will be key to succeeding in some of the other labs later in the course
Join a class Slack group to discuss OSINT and the class by way of a lab that walks you through the setup and use of the application
Section 2
Harvest web data such as Google Analytics IDs and the information within HTTPS certificates
Trace a home address and phone number to their owners
Gather email addresses for a company
Use a reconnaissance framework to rapidly scan websites looking for specific user accounts
Search reverse images to find the identity of the person and other places where that image was used
Section 3
Execute queries on search engines to find information about someone
Conduct Facebook queries to retrieve surface and deep data
Analyze tweets to determine sentiment and discover where the tweets are geolocated
Scrape metadata and map GPS coordinates
Section 4
Use online mapping sites to recon an area
Search for wireless network data and use it to verify an alibi
Run an OSINT framework to discover what information can be found about a domain
Examine various government websites to answer trivia questions
Gather data points about the CEO and the systems used at a business
Section 5
Dive into the deep web by using Tor to visit Internet sites and hidden services, and set up our own hidden service
Query the HaveIBeenPwned.com website and API to find compromised user accounts
Use translation sites to practice translating text into other languages
Discover the popular websites and mobile apps used in several countries
Undertake the Solo CTF that brings together many of the previous labs and helps students practice process
Section 6
Participate in the group Capture-the-Flag competition
Course Syllabus:
SEC487.1: Foundations of OSINT
Overview
We begin with the basics and answer the questions "what is OSINT" and "how do people use it." This first section of this course is about level-setting and ensuring that all students understand the background behind what we do in the OSINT field. We also establish the foundation for the rest of the course by learning how to document findings and set up an OSINT platform. This information taught in this section is a key component for the success of an OSINT analyst because without these concepts and processes in place, researchers can get themselves into serious trouble during assessments by inadvertently alerting their targets or improperly collecting data.
CPE/CMU Credits: 6
Topics
Course Introduction
Understanding OSINT
Goals of OSINT Collection
Diving into Collecting
Taking Excellent Notes
Determining Your Threat Profile
Setting up an OSINT Platform
Effective Habits and Process
Leveraging Search Engines
SEC487.2: Gathering, Searching, and Analyzing OSINT
Overview
OSINT data collection begins in section two after we get a glimpse of some of the fallacies that could influence our conclusions and recommendations. From this point in the course forward, we examine distinct categories of data and think about what it could mean for our investigations. Retrieving data from the Internet could mean using a web browser to view a page or, as we learn in this section, using command line tools, scripts, and helper applications.
CPE/CMU Credits: 6
Topics
Data Analysis Challenges
Harvesting Web Data
File Metadata Analysis
OSINT Frameworks
Basic Data: Addresses and Phone Numbers
Basic Data: Email Addresses
User Names
Avatars and Reverse Image Searches
Additional Public Data
Creating Sock Puppets
SEC487.3: Social Media, Geolocation, and Imagery
Overview
Section three kicks off by examining free and paid choices in people search engines and understanding how to use the data we receive from them. Some of these engines provide social media content in their results. This makes a terrific transition for us to move into social media data, geolocation, and eventually mapping and imagery.
CPE/CMU Credits: 6
Topics
People Search Engines
Exercise People Searching
Facebook Analysis
LinkedIn Data
Twitter Data
Geolocation
Imagery and Maps
SEC487.4: Networks, Government, and Business
Overview
Section four focuses on many different but related OSINT issues. This is our blue team day, as we dive into OSINT for IP addresses, domain names, DNS, and Whois. We then move into how to use wireless network information for OSINT. We end the section with two huge modules on searching international government websites for OSINT data and supporting business processes with OSINT.
CPE/CMU Credits: 6
Topics
Whois
IP Addresses
DNS
Finding Online Devices
Wireless Networks
Recon Tool Suites and Frameworks
Government Data
Researching Companies
SEC487.5: The Dark Web, Breach Data, and International Issues
Overview
The beginning of section five focuses on understanding and using three of the dark web networks. Students will learn why people use Freenet, I2P, and Tor. Each network is discussed at length so that students don't just know how and why to use it, but also gain an understanding of how those networks work. With the Tor network being such a big player in the dark web, the course spends extra time diving into its resources.
After tackling the dark web, we examine how we can use breach data in our cases and to address international OSINT issues. We end the section by examining how to find and track vehicles of all sizes.
The end of this section is a massive lab, the Solo Capture-the-Flag (CTF) Challenge that helps students put together all that they have learned up until now in the course. Through a semi-guided walk-through that touches on many of the concepts taught throughout the course, students complete a full OSINT assessment at their own speed. Setting aside time to work through our OSINT process in an organized manner reinforces key concepts and allows students to practice executing OSINT process, procedures, and techniques.
CPE/CMU Credits: 6
Topics
The Surface, Deep, and Dark Webs
The Dark Web
Freenet
I2P - Invisible Internet Project
Tor
Monitoring and Alerting
International Issues
Vehicle Searches
Solo CTF Challenge
SEC487.6: Capstone: Capture (and Present) the Flag
Overview
The capstone for the course is a group event that brings together everything that students have learned throughout the course. This is not a "canned" Capture-the-Flag event where specific flags are planted and your team must find them. It is a competition where each team will collect specific OSINT data about certain targets. The output from this work will be turned in as a "deliverable" to the "client" (the instructor and fellow classmates). This multi-hour, hands-on event will reinforce what the students practiced in the Solo CTF in the previous section before and add the complexity of performing OSINT assessments under pressure and in a group.
CPE/CMU Credits: 6
Topics
Capstone Capture-the-Flag Event
