{"id":23855,"date":"2018-05-07T14:14:42","date_gmt":"2018-05-07T14:14:42","guid":{"rendered":"https:\/\/www.checkmarx.com\/?p=23855"},"modified":"2024-07-28T06:58:30","modified_gmt":"2024-07-28T06:58:30","slug":"redos-go","status":"publish","type":"post","link":"https:\/\/checkmarx.com\/blog\/redos-go\/","title":{"rendered":"Diving Deep into Regular Expression Denial of Service (ReDoS) in Go"},"content":{"rendered":"

Go Programming Language (also known as Golang) is an open-source programming language created by Google. Go is compiled, is statically typed as in C (with garbage collection), with limited structural typing, memory safety features and CSP-style concurrent features.\u00a0In this blog post, we’ll recap Go\u2019s security posture facing Regular Expression Denial of Service (ReDoS) attacks. But first, let\u2019s start by explaining the concept of ReDoS and how such attacks can be exploited and mitigated. This blog post includes a set of practical examples using different programming languages, aiming to show how the Go implementation avoids ReDoS.
\n
\nThe topic of this report was motivated byongoing research on the topic of Go security, where we aim to discover vulnerabilities lurking in Go packages.
\nfunc sqli() {<\/span>
\nusername := r.Form.Get(“username”)<\/span>
\nsql := “SELECT * FROM user WHERE username='” + username + “‘” row_fullname := db.QueryRow(sql)<\/span>
\nfmt.Printf(“Welcome, %sn”, row_fullname)<\/span>
\n}<\/span>
\nListing 1: <\/strong>Golang SQLi example<\/span><\/p>\n

ReDoS<\/h2>\n

Regular Expression Denial of Service (ReDoS) is an algorithmic complexity attack that provokes a Denial of Service (DoS). ReDos attacks are caused by a regular expression that takes a very long time to be evaluated, exponentially related with the input size. This exceptionally long time in the evaluation process is due to the implementation of the regular expression in use, for example, recursive backtracking ones.
\nA regular expression, better known as a \u2018regex\u2019, is a sequence of characters that defines a search pattern, used to search for one or more characters within a string. One of the handy usages of a regex is information validation, i.e., ensuring that only properly formed data is being submitted.
\nFor example, let\u2019s pretend that we want to apply a regular expression over the username <\/em>input of listing 1. Thus, a simple regex could be:
\n\/^[a-zA-Z0-9_-]{3,10}$\/<\/span>
\nListing 2: <\/strong>Regex example 1<\/span>
\nThe regular expression is contained between the slash characters and in the pattern 2 <\/a>regex. We start by telling the parser to find the beginning of the string (\u02c6), followed by any lowercase letter (a-z), uppercase letter (A-Z), number (0-9), an underscore, or a hyphen. The {3,10} section makes sure that the entered string has a length between three and ten characters. Finally, the $ represents the end of the string.\u00a0For this regex, if we used the input \u201d<\/strong>checkmarx\u201d <\/strong>\u00a0it would match the pattern:<\/p>\n

\"\"

Figure 1: Example \u2013 Regex 1<\/p><\/div>\n

On the other hand, if we used a string like \u201d<\/strong>checkmarx\u2019 OR SLEEP(10)–\u201d<\/strong>\u00a0it would not match the pattern.<\/p>\n

\"\"

Figure 2: Example \u2013 Regex 1 Fail<\/p><\/div>\n

Evil Regular Expressions<\/h2>\n

Even though the benefits of using regexs for input validations are great, depending on the way they are written and the engine used, a malicious user can leverage it and make the application or service unavailable. Thus, evil regexs are the root cause of the ReDoS issue. They are considered evil or malicious if they can stuck on crafted input. To understand this better, let\u2019s consider the following regular expression:
\n\/A(B|C+)+D\/<\/span>
\nListing 3: <\/strong>Evil Regex example 1<\/span>
\nIn this scenario, this regex pattern starts by searching for the character \u2019A.\u2019 Then, the following string must either be the character \u2019B\u2019 or one or more \u2019C\u2019s, (B|C+). The next + indicates that it can search for one or more occurrences of the previous string. Finally, the \u2019D\u2019 ensures that the string is terminated by the character \u2019D\u2019. To match this regular expression, any input of the following type would be accepted:
\nABCD <\/span>
\nABCBD <\/span>
\nACD <\/span>
\nACBD<\/span>
\nListing 4: <\/strong>Valid input for Evil regex 1<\/span>
\nTo show the differences between the implementations used by different languages, we created simple programs in four different languages: Python, JavaScript, PHP and Go.\u00a0All created programs use the regex from example 3.\u00a0This benchmark was done incrementing passed inputs, allowing us to visually understand the different behaviors of the program depending on the variations of the inputs.
\nIn the example code from listing 5, we show a simple Python implementation to evaluate the evil regex. We start by testing the valid inputs from listing 4. Then we send malicious inputs to try to get the program stuck.
\nregex = r”A(B|C+)+D”<\/span>
\ntest_str = raw_input(“Enter the string: “)<\/span>
\nmatches = re.finditer(regex, test_str) (…)<\/span>
\nprint (“It took: %s seconds” % elapsedTime)<\/span>
\nListing 5: <\/strong>Python Regex compiler<\/span>
\nTo craft malicious inputs, we started by incrementing valid and invalid inputs, and tweaking it according to the time differences between them. At some point, we found some relevant discrepancies. In figure 4, we show the attempted malicious payloads and the matching elapsed time for evaluation. We see that when a malicious input payload of type AC+E, where + represents one or more occurrences of the character C, is sent with more than 20 Cs, the elapsed time starts to double for each new C.
\n\"\"
\nThe same principle was applied to the other languages and we maintained the input cases in order to compare the results. The next example is for JavaScript. Listing 6 shows the JS code snippet:
\nconst regex = \/A(B|C+)+D\/g; (…)<\/span>
\nlet m;<\/span>
\nwhile ((m = regex.exec(str)) !== null) { (…)<\/span>
\n} (…)<\/span>
\nconsole.log(“It took: ” + seconds + ” seconds”);<\/span>
\nListing 6: <\/strong>JS Regex compiler<\/span>
\nThe obtained results for the JavaScript implementation are as follows:
\n\"\"
\nThe next example is PHP:
\n$re = ‘\/A(B|C+)+D\/’;<\/span>
\npreg_match_all($re, $str, $matches, PREG_SET_ORDER, 0); (…)<\/span>
\necho $timediff.” seconds”;<\/span>
\nListing 7: <\/strong>PHP Regex compiler<\/span>
\nWhere the results from figures 7 and 8 produced these results:
\n\"\"
\nFinally, an example for Go. We created the following program:
\nfunc main() {<\/span>
\nre := regexp.MustCompile(`A(B|C+)+D`) (…)<\/span>
\nfor i, match := range re.FindAllString(str, -1) { fmt.Println(match, “found at index”, i)<\/span>
\n} (…)<\/span>
\nfmt.Println(“It took:”, elapsed.Seconds(), “seconds”)<\/span>
\n}<\/span>
\nListing 8: <\/strong>Go Regex compiler<\/span>
\nAnd produced the following results:
\n\"\"
\nWe can see that the results from PHP (8) and Go (10) implementations are much different from the Python (4) and JavaScript (6) results. The malicious inputs used in Python and JavaScript generated exponential time increments, versus the linear time responses for PHP and Go.
\nIn the PHP regex implementation, we used the Perl Compatible Regular Expressions (PCRE) library 1<\/a>, which uses\u00a0backreferences. The official regex package [2] implemented in Go uses the RE2 engine 2<\/a>, which does not support backreferences, and guarantees a linear time execution while avoiding regex denial of service.
\nIn table 1, we summarize the obtained results for each programming language, where it is displayed the elapsed time in seconds for each input that was tested. For the purpose of this post, we only tested ten inputs, starting with \u00a020 \u2019C\u2019 characters and incrementing one unit. This clarifies that for the Python and JavaScript implementations the time doubles when a new C is added.
\nFinally, in the chart seen in figure 11, it becomes visually clear what the discrepancies are between the results and places of the performances of PHP and Go side by side.
\n\"\"
\n\"\"<\/p>\n

Behind the Curtains<\/h2>\n

The most commonly used algorithms to implement regular expression matching are:<\/p>\n