File: scripts/Generate_CheatSheets_TOC.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Python3 script to generate the summary markdown page that is used
by GitBook to generate the offline website.

The summary markdown page is named "TOC.md" and is generated in the
same location that the script in order to be moved later by the caller script.
"""
import os

# Define templates
cs_md_link_template = "* [%s](cheatsheets/%s)"

# Scan all CS files
cheatsheets = [f.name for f in os.scandir("../cheatsheets") if f.is_file()]
cheatsheets.sort()

# Generate the summary file
with open("TOC.md", "w") as index_file:
    index_file.write("# Summary\n\n")
    index_file.write("### Cheatsheets\n\n")
    index_file.write(cs_md_link_template % ("Index Alphabetical", "Index.md"))
    index_file.write("\n")
    index_file.write(cs_md_link_template % ("Index ASVS", "IndexASVS.md"))
    index_file.write("\n")
    index_file.write(cs_md_link_template % ("Index ASVS", "IndexMASVS.md"))
    index_file.write("\n")
    index_file.write(cs_md_link_template % ("Index Proactive Controls", "IndexProactiveControls.md"))
    index_file.write("\n")
    for cheatsheet in cheatsheets:
        if cheatsheet != "Index.md" and cheatsheet != "IndexASVS.md" and cheatsheet != "IndexMASVS.md" and cheatsheet != "IndexProactiveControls.md" and cheatsheet != "TOC.md":
            cs_name = cheatsheet.replace("_"," ").replace(".md", "").replace("Cheat Sheet", "")
            index_file.write(cs_md_link_template % (cs_name, cheatsheet))
            index_file.write("\n")
print("Summary markdown page generated.")


File: scripts/Identify_Old_Issue_And_PR.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Python3 script to identify any Issue or PR meeting the following criteria:
- For Issue (Comments of the issue do not contain the info when a PR is referenced):
    - Has assignees
    - Has not the label HELP_WANTED or INTERNAL
    - Has not been updated since more than 1 month
- For PR:
    - Has the label WAITING_UPDATE
    - Has not been updated since more than 1 month

Do not require to have a local copy of the GitHub repository.

Dependencies: pip install requests
"""
import sys
import requests
import json
from datetime import datetime

# Define constants
## API to retrieve the list of Issues/PR (GitHub REST API v3 considers every pull request an issue, but not every issue is a pull request)
## Ask to the API to sort the list by the updated date in order to have the oldest on the top on the list
## See https://developer.github.com/v3/issues/#list-issues-for-a-repository for explanation
ISSUE_API = "https://api.github.com/repos/OWASP/CheatSheetSeries/issues?page=1&per_page=1000&sort=updated&direction=asc"
## Expiration delay
MAX_MONTHS_ALLOWED = 1

# Define utility function: Cf criteria in the comment of the script for the criteria
def is_old_issue(issue):
    has_assignees = (len(issue["assignees"]) > 0)
    has_help_wanted_label = False
    has_internal_label = False
    labels = issue["labels"]
    for label in labels:
        if label["name"] == "HELP_WANTED":
            has_help_wanted_label = True
        elif label["name"] == "INTERNAL":
            has_internal_label = True
    return has_assignees and (not has_help_wanted_label and not has_internal_label)

def is_old_pull_request(issue):
    has_waiting_for_update_label = False
    labels = issue["labels"]
    for label in labels:
        if label["name"] == "WAITING_UPDATE":
            has_waiting_for_update_label = True
            break
    return has_waiting_for_update_label

# Grab the list of open Issues/PR
buffer = "Grab the list of open Issues/PR via the GitHub API...\n"
response = requests.get(ISSUE_API)
if response.status_code != 200:
    print("Cannot load the list of Issues/PR content: HTTP %s received!" % response.status_code)
    sys.exit(1)
issues = response.json()

# Process the obtained list
buffer += "Process the obtained list (%s items)...\n" % len(issues)
issues = response.json()
old_issues = {"PR":[], "ISSUE":[]}
for issue in issues:
    # Date format is 2019-08-24T15:29:55Z
    last_update = datetime.strptime(issue["updated_at"], "%Y-%m-%dT%H:%M:%SZ")
    diff_in_months = round(abs((datetime.today() - last_update).days / 30))
    if diff_in_months > MAX_MONTHS_ALLOWED:
        id = str(issue["number"])
        if "pull_request" in issue and is_old_pull_request(issue):
            old_issues["PR"].append(id)
        elif is_old_issue(issue):
            old_issues["ISSUE"].append(id)

# Render the result
if (len(old_issues["PR"]) + len(old_issues["ISSUE"])) != 0:
    buffer += "State:\n"
    if len(old_issues["PR"]) > 0:
        buffer += "Old pull request identified (%s items): %s\n" % (len(old_issues["PR"]), " / ".join(old_issues["PR"]))
    if len(old_issues["ISSUE"]) > 0:
        buffer += "Old issue identified (%s items): %s\n" % (len(old_issues["ISSUE"]), " / ".join(old_issues["ISSUE"]))
else:
    buffer += "State: Nothing identified!"
print(buffer)

# Send notification the project management channel on Slack if the url of the webhook is passed as unique first parameter
if len(sys.argv) == 2:
    if (len(old_issues["PR"]) + len(old_issues["ISSUE"])) == 0:
        color = "good"
    else:
        color = "warning"
    message = "{\"text\": \"Old PR and Issue identification watchdog\",\"attachments\": [ {\"fallback\": \"%s\",\"color\":\"%s\",\"title\": \"Status\",\"text\": \"%s\"}]}" % (color, buffer, buffer)
    request_headers = {"Content-Type": "application/json"}
    response = requests.post(sys.argv[1], headers=request_headers, data=message)
    if response.status_code != 200:
        print("Cannot send notification to slack: HTTP %s received!" % response.status_code)
        sys.exit(2)


File: scripts/Generate_RSS_Feed.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Python3 script to generate an RSS feed XML file based on merged pull requests:
See https://github.com/OWASP/CheatSheetSeries/issues/186
Do not require to have a local copy of the GitHub repository.
Dependencies: pip install requests feedgen
"""
import sys
import json
from datetime import datetime

import requests
from feedgen.feed import FeedGenerator

# Define constants
# API to retrieve the list of PR
# See https://developer.github.com/v3/pulls/#list-pull-requests for explanation
PR_API = "https://api.github.com/repos/OWASP/CheatSheetSeries/pulls?page=1&per_page=1000&state=closed"

# Grab the list of open PR
print("[+] Grab the list of closed PR via the GitHub API...")
response = requests.get(PR_API)
if response.status_code != 200:
    print("Cannot load the list of PR content: HTTP %s received!" %  response.status_code)
    sys.exit(1)
pull_requests = response.json()

# Process the obtained list and generate the feed in memory
print("[+] Process the obtained list and generate the feed in memory (%s) items)..." % len(pull_requests))
feed_generator = FeedGenerator()
current_date = datetime.utcnow().strftime("%a, %d %B %Y %H:%M:%S GMT")  # Sun, 19 May 2002 15:21:36 GMT
feed_generator.id("https://cheatsheetseries.owasp.org/")
feed_generator.title("OWASP Cheat Sheet Series update")
feed_generator.description("List of the last updates on the content")
feed_generator.author({"name": "Core team", "email": "dominique.righetto@owasp.org"})
feed_generator.link({"href": "https://cheatsheetseries.owasp.org", "rel": "self"})
feed_generator.link({"href": "https://github.com/OWASP/CheatSheetSeries", "rel": "alternate"})
feed_generator.language("en")
feed_generator.icon("https://cheatsheetseries.owasp.org/gitbook/images/favicon.ico")
feed_generator.pubDate(current_date)
feed_generator.lastBuildDate(current_date)
for pull_request in pull_requests:
    # Take only merged PR
    if pull_request["merged_at"] is None:
        continue
    # Convert merge date from 2019-08-25T06:36:35Z To Sun, 19 May 2002 15:21:36 GMT
    merge_date_src = pull_request["merged_at"]
    merge_date_dst = datetime.strptime(merge_date_src, "%Y-%m-%dT%H:%M:%SZ").strftime("%a, %d %B %Y %H:%M:%S GMT")
    feed_entry = feed_generator.add_entry()
    feed_entry.id(pull_request["html_url"])
    feed_entry.title(pull_request["title"])
    feed_entry.link({"href": pull_request["html_url"], "rel": "self"})
    feed_entry.link({"href": pull_request["html_url"], "rel": "alternate"})
    feed_entry.pubDate(merge_date_dst)
    feed_entry.updated(merge_date_dst)
    contributors = []
    for assignee in pull_request["assignees"]:
        contributors.append({"name": assignee["login"], "uri": "https://github.com/%s" % assignee['login']})
    feed_entry.contributor(contributors)

# Save the feed to a XML file
print("[+] Save the feed to a XML file...")
feed_generator.atom_file("News.xml")
print("[+] Feed saved to 'News.xml'.")



File: scripts/Update_CheatSheets_Index.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Python3 script to generate the index markdown page that
reference all cheat sheets grouped by the first letter.

The index markdown page is located on the root folder
and is named "Index.md".
"""
import os
from collections import OrderedDict

# Define utility functions
def extract_languages_snippet_provided(cheatsheet):
    languages = []
    markers = ["javascript", "java", "csharp", "c", "cpp", "html", "xml", "python",
               "ruby", "php", "json", "sql", "bash", "shell", "coldfusion", "perl",
               "vbnet"]
    with open("../cheatsheets/" + cheatsheet, encoding="utf8") as cs_file:
        cs_content = cs_file.read().lower().replace(" ","")
    for marker in markers:
        if "```" + marker + "\n" in cs_content:
            languages.append(marker.capitalize())
    return languages

# Define templates
cs_md_link_template = "[%s](cheatsheets/%s)"
language_md_link_template = "![%s](assets/Index_%s.svg)"
header_template = "## %s\n\n"
top_menu_template = "[%s](Index.md#%s)"
cs_count_template = "**%s** cheat sheets available."
cs_index_title_template = "# Index Alphabetical\n\n"

# Scan all CS files
index = {}
cs_count = 0
cheatsheets = [f.name for f in os.scandir("../cheatsheets") if f.is_file()]
for cheatsheet in cheatsheets:
    letter = cheatsheet[0].upper()
    if letter not in index:
        index[letter] = [cheatsheet]
    else:
        index[letter].append(cheatsheet)
    cs_count += 1
index = OrderedDict(sorted(index.items()))

# Generate the index file
with open("../Index.md", "w", encoding="utf-8") as index_file:
    index_file.write(cs_index_title_template)
    index_count = len(index)
    index_file.write(cs_count_template % cs_count)
    index_file.write("\n\n*Icons beside the cheat sheet name indicate in which language(s) code snippet(s) are provided.*")
    index_file.write("\n\n")
    # Generate the top menu
    for letter in index:
        index_file.write(top_menu_template % (letter, letter.lower()))
        index_file.write(" ")
    index_file.write("\n\n")
    # Generate letter sections
    j = 0
    for letter in index:
        cs_count =  len(index[letter])
        index_file.write(header_template % letter)
        i = 0
        for cs_file in index[letter]:
            cs_name = cs_file.replace("_", " ").replace(".md", "").strip()
            index_file.write(cs_md_link_template % (cs_name, cs_file))
            languages = extract_languages_snippet_provided(cs_file)
            if len(languages) > 0:
                index_file.write(" ")
                for language in languages:
                    index_file.write(language_md_link_template % (language, language))
                    index_file.write(" ")
            i += 1
            index_file.write("\n")
            if i != cs_count:
                index_file.write("\n")
        j += 1
        if j != index_count:
            index_file.write("\n")

# Clean trailing whitespaces
with open("../Index.md", "r", encoding="utf-8") as file:
    cleaned_lines = [line.rstrip() + "\n" for line in file]

with open("../Index.md", "w", encoding="utf-8") as file:
    file.writelines(cleaned_lines)

print("Index updated.")



File: scripts/Generate_Technologies_JSON.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Python3 script to generate a JSON structure with the list of
all cheatsheets classified by the technology used in the samples
of code provided using the alphabetical index as source:
https://raw.githubusercontent.com/OWASP/CheatSheetSeries/master/Index.md

Do not require to have a local copy of the GitHub repository.

Dependencies: pip install requests
"""
import sys
import requests
import json
from collections import OrderedDict

# Define templates
CS_BASE_URL = "https://cheatsheetseries.owasp.org/cheatsheets/%s.html"

# Grab the index MD source from the GitHub repository
response = requests.get(
    "https://raw.githubusercontent.com/OWASP/CheatSheetSeries/master/Index.md")
if response.status_code != 200:
    print("Cannot load the INDEX content: HTTP %s received!" %
          response.status_code)
    sys.exit(1)
else:
    data = OrderedDict({})
    for line in response.text.split("\n"):
        if "(assets/Index_" in line:
            work = line.strip()
            # Extract the name of the CS
            cs_name = work[1:work.index("]")]
            # Extract technologies and map the CS to them
            technologies = work.split("!")[1:]
            for technology in technologies:
                technology_name = technology[1:technology.index("]")].upper()
                if technology_name not in data:
                    data[technology_name] = []
                data[technology_name].append(
                    {"CS_NAME": cs_name, "CS_URL": CS_BASE_URL % cs_name.replace(" ", "_")})
    # Display the built structure and formatted JSON
    print(json.dumps(data, sort_keys=True, indent=1))
    sys.exit(0)



# Welcome to the OWASP Cheat Sheet Series

[![OWASP Flagship](https://img.shields.io/badge/owasp-flagship%20project-48A646.svg)](https://www.owasp.org/index.php/OWASP_Project_Inventory#tab=Flagship_Projects)
[![Creative Commons License](https://img.shields.io/github/license/OWASP/CheatSheetSeries)](https://creativecommons.org/licenses/by-sa/4.0/ "CC BY-SA 4.0")

Welcome to the official repository for the Open Web Application Security Project® (OWASP) Cheat Sheet Series project. The project focuses on providing good security practices for builders in order to secure their applications.

In order to read the cheat sheets and **reference** them, use the project [official website](https://cheatsheetseries.owasp.org). The project details can be viewed on the [OWASP main website](https://owasp.org/www-project-cheat-sheets/) without the cheat sheets.

:triangular_flag_on_post: Markdown files are the working sources and aren't intended to be referenced in any external documentation, books or websites.

## Cheat Sheet Series Team

### Project Leaders

- [Jim Manico](https://github.com/jmanico)
- [Jakub Maćkowski](https://github.com/mackowski)

### Core Team

- [Kevin W. Wall](https://github.com/kwwall)
- [Shlomo Zalman Heigh](https://github.com/szh)

## Chat With Us

We're easy to find on Slack:

1. Join the OWASP Group Slack with this [invitation link](https://owasp.org/slack/invite).
2. Join the [#cheatsheets channel](https://owasp.slack.com/messages/C073YNUQG).

Feel free to ask questions, suggest ideas, or share your best recipes.

## Contributions, Feature Requests, and Feedback

We are actively inviting new contributors! To start, please read the [contribution guide](CONTRIBUTING.md).

This project is only possible thanks to the work of many dedicated volunteers. Everyone is encouraged to help in ways large and small. Here are a few ways you can help:

- Read the current content and help us fix any spelling mistakes or grammatical errors.
- Choose an existing [issue](https://github.com/OWASP/CheatSheetSeries/issues) on GitHub and submit a pull request to fix it.
- Open a new issue to report an opportunity for improvement.

### Automated Build

This [link](https://cheatsheetseries.owasp.org/bundle.zip) allows you to download a build (ZIP archive) of the offline website.

### Local Build [![pyVersion3x](https://img.shields.io/badge/python-3.x-blue.svg)](https://www.python.org/downloads/)

The OWASP Cheat Sheet Series website can be built and tested locally by issuing the following commands:

```sh
make install-python-requirements
make generate-site
make serve  # Binds port 8000
```

### Container Build

The OWASP Cheat Sheet Series website can be built and tested locally inside a container by issuing the following commands:

#### Docker

```sh
docker build -t cheatsheetseries .
docker run --name cheatsheetseries -p 8000:8000 cheatsheetseries
```

#### Podman

```sh
podman build -t cheatsheetseries .
podman run --name cheatsheetseries -p 8000:8000 localhost/cheatsheetseries
```

## Contributors

- **From 2014 to 2018:** [V1](CONTRIBUTOR-V1.md) - Initial version of the project hosted on the [OWASP WIKI](https://wiki.owasp.org).
- **From 2019:** [V2](https://github.com/OWASP/CheatSheetSeries/graphs/contributors) - Hosted on [GitHub](https://github.com/OWASP/CheatSheetSeries).

## Special thanks

A special thank you to the following people for their help provided during the migration:

- [Dominique Righetto](https://github.com/righettod): For his special leadership and guidance.
- [Elie Saad](https://github.com/ThunderSon): For valuable help in updating the OWASP Wiki links for all the migrated cheat sheets and for years of leadership and other project support.
- [Jakub Maćkowski](https://github.com/mackowski): For valuable help in updating the OWASP Wiki links for all the migrated cheat sheets.

Open Web Application Security Project and OWASP are registered trademarks of the OWASP Foundation, Inc.
