Skip to content

README.md

Latest commit

 

History

History
57 lines (37 loc) · 1.56 KB

README.md

File metadata and controls

57 lines (37 loc) · 1.56 KB
Raw

Open Source Vulnerabilities Database Tool

🔗 https://osv.dev/list

OSV is the Open Source Vulnerability Database provided by Google. All the resulting datasets are available at ~/data/osv/ (in case you want to check it).

1. Getting the raw data

The raw dataset is not available in this repository. There are two means of getting the dataset:

Run the scripts to collect the dataset
  1. Set up a GitHub Token. First, copy the github template file:
cp config/github_template.json config/github.json
  1. Change the placeholder to a personal github token.
{
	"token": "TOKEN_VALUE"
}
  1. Run the following bash script to collect the vulnerabilities from the different ecosystems (GHSA, DWF, Go, Linux, Maven, NuGet, OSS-Fuzz, PyPI, RubyGems, crates.io, npm, Hex Packagist):
source download.sh

This script will retrieve and process all the data for the different ecosystems.

Download our mirror

The dataset is available through Google Drive. Download our google drive mirror by running the following command:

gdown https://drive.google.com/uc?id=1_n08wgdyEFT-j53f_gzUpDbQBMMFDqKW

2. Generate Dataset

Filter the entries with references to commits in source code hosting websites such as github, bitbucket, gitlab and git.

  1. Merge, plot stats and normalize OSV data:
source generate_data.sh
  1. Filter OSV data by source code hosting website (github, bitbucket, gitlab or git):
source filter_data_by_source.sh github