WebDriver getCurrentUrl() returning malformed URI

Question

I'm involved in writing a (Java/Groovy) browser-automation app with Selenium 2 and FireFox driver.

Currently there is an issue with some URLs we find in the wild that are apparently using bad URI syntax. (specifically curly braces ({}), |'s and ^'s).

String url = driver.getCurrentUrl(); // http://example.com/foo?key=val|with^bad{char}acters

When trying to construct a java.net.URI from the string returned by driver.getCurrentUrl() a URISyntaxException is thrown.

new URI(url); // java.net.URISyntaxException: Illegal character in query at index ...

Encoding the whole url before constructing the URI will not work (as I understand it).

The whole url is encoded, and it doesn't preseve any pieces of it that I can parse in any normal fashion. For example, with this uri-safe string, URI can't know the difference between a & as the query-string-param delimeter or %26 (its encoded value) in the content of a single qs-param.

String encoded = URLEncoder.encode(url, "UTF-8") // http%3A%2F%2Fexample.com%2Ffoo%3Fkey%3Dval%7Cwith%5E%7Cbad%7Ccharacters
URI uri = new URI(encoded)
URLEncodedUtils.parse(uri, "UTF-8") // []

Currently the solution is, before constructing the URI, running the following (groovy) code:

["|", "^", "{", "}"].each {
    url = url.replace(it, URLEncoder.encode(it, "UTF-8"))
}

But this seems dirty and wrong.

I guess my question is multi-part:

1. Why does FirefoxDriver return a String rather than a URI?
2. Why is this String malformed?
3. What is best practice for dealing with this kind of thing?

Answer 1

We can partially encode query string parameters, as discussed in comments, it should work.

Other way is to use galimatias library:

import io.mola.galimatias.GalimatiasParseException;
import io.mola.galimatias.URL;

import java.net.URI;
import java.net.URISyntaxException;

public class Main {

    public static void main(String[] args) throws URISyntaxException {
        String example1 = "http://example.com/foo?key=val-with-a-|-in-it";
        String example2 = "http://example.com?foo={bar}";

        try {
            URL url1 = URL.parse(example1);
            URI uri1 = url1.toJavaURI();
            System.out.println(url1);
            System.out.println(uri1);

            URL url2 = URL.parse(example2);
            URI uri2 = url2.toJavaURI();
            System.out.println(url2);
            System.out.println(uri2);
        } catch (GalimatiasParseException ex) {
            // Do something with non-recoverable parsing error
        }
    }
}

Output:

http://example.com/foo?key=val-with-a-|-in-it
http://example.com/foo?key=val-with-a-%7C-in-it
http://example.com/?foo={bar}
http://example.com/?foo=%7Bbar%7D

Answer 2

driver.getCurrentUrl() gets a string from the browser and before making it into an URL, you should URL encode the string.

See Java URL encoding of query string parameters for an example of this in Java.

Answer 3

Will this work for you?

import java.net.URI;
import java.net.URL;
import java.net.URLEncoder;


public class Sample {

public static void main(String[] args) throws UnsupportedEncodingException {
    String urlInString="http://example.com/foo?key=val-with-a-{-in-it";
    String encodedURL=URLEncoder.encode(urlInString, "UTF-8");

    URI encodedURI=URI.create(encodedURL);
    System.out.println("Actual URL:"+urlInString);
    System.out.println("Encoded URL:"+encodedURL);
    System.out.println("Encoded URI:"+encodedURI);

}

}

Output:

Actual URL:http://example.com/foo?key=val-with-a-{-in-it Encoded URL:http%3A%2F%2Fexample.com%2Ffoo%3Fkey%3Dval-with-a-%7B-in-it Encoded URI:http%3A%2F%2Fexample.com%2Ffoo%3Fkey%3Dval-with-a-%7B-in-it

Answer 4

Another Solution is to split the URL fetched and then use them to create the URL you want. This will ensure that you get all the features of URL class.

import java.io.UnsupportedEncodingException;
import java.net.MalformedURLException;
import java.net.URI;     
import java.net.URISyntaxException;      
import java.net.URL;

public class Sample {

public static void main(String[] args) throws UnsupportedEncodingException,
        URISyntaxException, MalformedURLException {
    String uri1 = "http://example.com/foo?key=val-with-a-{-in-it";

    String scheme=uri1.split(":")[0];

    String authority=uri1.split("//")[1].split("/")[0];

    String path=uri1.split("//")[1].split("/")[1].split("\\?")[0];  

    String query=uri1.split("\\?")[1];  


    URI uri = null;
    uri = new URI(scheme, authority, "/"+path, query,null);

    URL url = null;

    url = uri.toURL();

    System.out.println("URI's Query:"+uri.getQuery());
    System.out.println("URL's Query:"+url.getQuery());

}

}