HTTPS
(Redirected from HTTPS Request)
Send a GET request to obtain the resource located at the URL "https://www.w3.org/", then print it to the console.
Checking the host certificate for validity is recommended.
Do not authenticate. That is the subject of other tasks.
Readers may wish to contrast with the HTTP Request task, and also the task on HTTPS request with authentication.
You are encouraged to solve this task according to the task description, using any language you may know.
- Task
Exactly the same as the HTTP task, assuming you compiled AWS with openssl support.
with AWS.Client;
with AWS.Response;
with Ada.Text_IO; use Ada.Text_IO;
procedure GetHttps is
begin
Put_Line (AWS.Response.Message_Body (AWS.Client.Get (
URL => "https://sourceforge.net/")));
end GetHttps;
print read "https://www.w3.org/"
URL := "https://sourceforge.net/"
WININET_Init()
msgbox % html := UrlGetContents(URL)
WININET_UnInit()
return
#include urlgetcontents.ahk
#include wininet.ahk
This code requires BaCon 3.8.2 or later.
OPTION TLS TRUE
website$ = "www.google.com"
OPEN website$ & ":443" FOR NETWORK AS mynet
SEND "GET / HTTP/1.1\r\nHost: " & website$ & "\r\n\r\n" TO mynet
WHILE WAIT(mynet, 1000)
RECEIVE dat$ FROM mynet
total$ = total$ & dat$
IF REGEX(dat$, "\r\n\r\n$") THEN BREAK : ' Quit receiving data when end indicator was reached
WEND
CLOSE NETWORK mynet
PRINT REPLACE$(total$, "\r\n[0-9a-fA-F]+\r\n", "\r\n", TRUE) : ' Remove chunk indicators from HTML data
:: Must have curl.exe
curl.exe -k -s -L https://sourceforge.net/
' ============================================
' https://rosettacode.org/wiki/HTTPS
' BazzBasic: https://github.com/EkBass/BazzBasic
' ============================================
LET response$ = HTTPGET("https://ekbass.github.io/BazzBasic/")
PRINT response$
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
CURL *curl;
char buffer[CURL_ERROR_SIZE];
int main(void) {
if ((curl = curl_easy_init()) != NULL) {
curl_easy_setopt(curl, CURLOPT_URL, "https://sourceforge.net/");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, buffer);
if (curl_easy_perform(curl) != CURLE_OK) {
fprintf(stderr, "%s\n", buffer);
return EXIT_FAILURE;
}
curl_easy_cleanup(curl);
}
return EXIT_SUCCESS;
}
using System;
using System.Net;
class Program
{
static void Main(string[] args)
{
var client = new WebClient();
var data = client.DownloadString("https://www.google.com");
Console.WriteLine(data);
}
}
This does not work for urls requiring a secure (SSL) connection.
Using the duck-streams as a convenient wrapper for Java's networking classes, grabbing the contents of an HTTPS URL is as easy as:
(use '[clojure.contrib.duck-streams :only (slurp*)])
(print (slurp* "https://sourceforge.net"))
The usual Java mechanisms can be used to manage acceptance of SSL certificates if required.
(print (slurp "https://sourceforge.net"))
Using curl
import std.stdio;
import std.net.curl;
auto data = get("https://sourceforge.net");
writeln(data);
First grabbing the entire body as a string, and then by pulling from a stream. This is the same code as in HTTP Request; drakma:http-request supports SSL.
(defun wget-drakma-string (url &optional (out *standard-output*))
"Grab the body as a string, and write it to out."
(write-string (drakma:http-request url) out))
(defun wget-drakma-stream (url &optional (out *standard-output*))
"Grab the body as a stream, and write it to out."
(loop with body = (drakma:http-request url :want-stream t)
for line = (read-line body nil nil)
while line do (write-line line)
finally (close body)))
;; Use
(wget-drakma-stream "https://sourceforge.net")
(format t "~a~%" (nth-value 0 (dex:get "https://www.w3.org/")))
program ShowHTTPS;
{$APPTYPE CONSOLE}
uses IdHttp, IdSSLOpenSSL;
var
s: string;
lHTTP: TIdHTTP;
begin
lHTTP := TIdHTTP.Create(nil);
try
lHTTP.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(lHTTP);
lHTTP.HandleRedirects := True;
s := lHTTP.Get('https://sourceforge.net/');
Writeln(s);
finally
lHTTP.Free;
end;
end.
The httpfs extension is standard and has only to be installed once. It can be used for both http and https requests.
install httpfs; -- if required
load httpfs;
from read_text( 'https://www.w3.org/' );
- Output:
(elided)
┌─────────────────────┬────────────────────────────────────────────────────┬───────┬─────────────────────┐ │ filename │ content │ size │ last_modified │ │ varchar │ varchar │ int64 │ timestamp │ ├─────────────────────┼────────────────────────────────────────────────────┼───────┼─────────────────────┤ │ https://www.w3.org/ │ <!doctype html>\n<html dir="ltr" lang="en" class… │ 56683 │ 2003-10-05 11:26:56 │ └─────────────────────┴────────────────────────────────────────────────────┴───────┴─────────────────────┘
file->string usage: the server must allow cross-domain access, or a browser add-on like cors-everywhere must be installed to bypass cross-domain checking.
;; asynchronous call back definition
(define (success name text) (writeln 'Loaded name) (writeln text))
;;
(file->string success "https:/sourceforge.net")
Synchronous
-module(main).
-export([main/1]).
main([Url|[]]) ->
inets:start(),
ssl:start(),
case http:request(get, {URL, []}, [{ssl,[{verify,0}]}], []) of
{ok, {_V, _H, Body}} -> io:fwrite("~p~n",[Body]);
{error, Res} -> io:fwrite("~p~n", [Res])
end.
Asynchronous
-module(main).
-export([main/1]).
main([Url|[]]) ->
inets:start(),
ssl:start(),
http:request(get, {Url, [] }, [{ssl,[{verify,0}]}], [{sync, false}]),
receive
{http, {_ReqId, Res}} -> io:fwrite("~p~n",[Res]);
_Any -> io:fwrite("Error: ~p~n",[_Any])
after 10000 -> io:fwrite("Timed out.~n",[])
end.
Using it
|escript ./req.erl https://sourceforge.net/
ELENA 6.x (using MbedTLS library) :
import net'http;
import mbedtls'client'registration;
public Program()
{
using(HttpClient client := HttpClient.open("https://www.google.com"))
{
HttpResponse response := client.get();
string content := response.readAsString();
Console.writeLine(content);
};
}The underlying .NET classes handle secure web connections the same way they manage insecure connections.
#light
let wget (url : string) =
let c = new System.Net.WebClient()
c.DownloadString(url)
#include once "windows.bi"
#include once "win/wininet.bi"
' Function to check the host certificate for validity
Function CheckCertificate(hRequest As HINTERNET) As Boolean
Dim As DWORD dwFlags, dwBuffLen = Sizeof(DWORD)
If InternetQueryOption(hRequest, INTERNET_OPTION_SECURITY_FLAGS, @dwFlags, @dwBuffLen) Then
If (dwFlags And SECURITY_FLAG_IGNORE_UNKNOWN_CA) = 0 Then Return True
End If
Return False
End Function
' Main program
Dim As HINTERNET hInternet, hConnect, hRequest
Dim As String url = "https://www.w3.org/"
Dim As String buffer
Dim As DWORD bytesRead
hInternet = InternetOpen("FreeBASIC HTTP Request", INTERNET_OPEN_TYPE_DIRECT, null, null, 0)
If hInternet Then
hConnect = InternetConnect(hInternet, "www.w3.org", INTERNET_DEFAULT_HTTPS_PORT, null, null, INTERNET_SERVICE_HTTP, 0, 0)
If hConnect Then
hRequest = HttpOpenRequest(hConnect, "GET", "/", null, null, null, INTERNET_FLAG_SECURE, 0)
If hRequest Then
If HttpSendRequest(hRequest, null, 0, null, 0) Then
If CheckCertificate(hRequest) Then
Do
buffer = Space(1024)
If InternetReadFile(hRequest, Strptr(buffer), Len(buffer), @bytesRead) = 0 Then Exit Do
If bytesRead = 0 Then Exit Do
Print Left(buffer, bytesRead);
Loop
Else
Print "Invalid certificate."
End If
Else
Print "Failed to send request."
End If
InternetCloseHandle(hRequest)
Else
Print "Failed to open request."
End If
InternetCloseHandle(hConnect)
Else
Print "Failed to connect."
End If
InternetCloseHandle(hInternet)
Else
Print "Failed to open internet."
End If
Sleep
print[read["https://sourceforge.net/"]]include "NSLog.incl"
local fn GET_HTTPS
CFStringRef response = unix @"curl -ksL https://sourceforge.net/"
CFDataRef dta = fn StringData( response, NSUTF8StringEncoding )
CFDictionaryRef options = @{NSDocumentTypeDocumentAttribute: NSHTMLTextDocumentType, NSCharacterEncodingDocumentAttribute: @(NSUTF8StringEncoding)}
CFAttributedStringRef aStr = fn AttributedStringWithHTML( dta, options )
NSLog( @"%@", aStr )
end fn
fn GET_HTTPS
HandleEvents- Output:
We're sorry -- the Sourceforge site is currently in Disaster Recovery mode. Please check back later. {
NSColor = "sRGB IEC61966-2.1 colorspace 0 0 0 1";
NSFont = "\"Times-Roman 12.00 pt. P [] (0x7f8f94e11ce0) fobj=0x7f8f94e0fda0, spc=3.00\"";
NSKern = 0;
NSParagraphStyle = "Alignment 4, LineSpacing 0, ParagraphSpacing 0, ParagraphSpacingBefore 0, HeadIndent 0, TailIndent 0, FirstLineHeadIndent 0, LineHeight 0/0, LineHeightMultiple 0, LineBreakMode 0, Tabs (\n), DefaultTabInterval 36, Blocks (\n), Lists (\n), BaseWritingDirection 0, HyphenationFactor 0, TighteningForTruncation YES, HeaderLevel 0 LineBreakStrategy 0";
NSStrokeColor = "sRGB IEC61966-2.1 colorspace 0 0 0 1";
NSStrokeWidth = 0;
}
package main
import (
"io"
"log"
"net/http"
"os"
)
func main() {
r, err := http.Get("https://sourceforge.net/")
if err != nil {
log.Fatalln(err)
}
io.Copy(os.Stdout, r.Body)
}
new URL("https://sourceforge.net").eachLine { println it }
This is just the example from Network.HTTP.Conduit, with the http URL replaced with an https one, since http-conduit natively supports https without needing any additional work.
#!/usr/bin/runhaskell
import Network.HTTP.Conduit
import qualified Data.ByteString.Lazy as L
import Network (withSocketsDo)
main = withSocketsDo
$ simpleHttp "https://sourceforge.net/" >>= L.putStr
Icon and Unicon
# Requires Unicon version 13
procedure main(arglist)
url := (\arglist[1] | "https://sourceforge.net/")
w := open(url, "m-") | stop("Cannot open " || url)
while write(read(w))
close(w)
end
- Output:
prompt$ unicon -s https.icn -x | head -n2 <!doctype html> <!-- Server: sfs-consume-15 -->
connection = URL new("https://sourceforge.net") openConnection
scanner = Scanner new(connection getInputStream)
while(scanner hasNext,
scanner next println
)
Using gethttp from Web Scraping
#page=: gethttp'https://sourceforge.net'
0
#page=: '--no-check-certificate' gethttp'https://sourceforge.net'
900
(We can not load the example page using https unless we disable certificate checking. The numbers are the number of characters retrieved.)
javax.net.ssl
Additional certificate information is available through the javax.net.ssl.HttpsURLConnection interface.
URL url = new URL("https://sourceforge.net");
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
Scanner scanner = new Scanner(connection.getInputStream());
while (scanner.hasNext()) {
System.out.println(scanner.next());
}
java.net.http
Using the standard Java 11 HTTP Client
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.Charset;
public class Main {
public static void main(String[] args) {
var request = HttpRequest.newBuilder(URI.create("https://sourceforge.net"))
.GET()
.build();
HttpClient.newHttpClient()
.sendAsync(request, HttpResponse.BodyHandlers.ofString(Charset.defaultCharset()))
.thenApply(HttpResponse::body)
.thenAccept(System.out::println)
.join();
}
}
Browser
Using fetch API and async/await:
const response = await fetch('https://rosettacode.org');
const text = await response.text();
console.log(text);
fetch("https://sourceforge.net").then(function (response) {
return response.text();
}).then(function (body) {
return body;
});
Node.js
require("https").get("https://sourceforge.net", function (resp) {
let body = "";
resp.on("data", function (chunk) {
body += chunk;
});
resp.on("end", function () {
console.log(body);
});
}).on("error", function (err) {
console.error("Error: " + err.message);
});
# v0.6.0
using Requests
str = readstring(get("https://sourceforge.net/"))
// version 1.1.2
import java.net.URL
import javax.net.ssl.HttpsURLConnection
import java.io.InputStreamReader
import java.util.Scanner
fun main(args: Array<String>) {
val url = URL("https://en.wikipedia.org/wiki/Main_Page")
val connection = url.openConnection() as HttpsURLConnection
val isr = InputStreamReader(connection.inputStream)
val sc = Scanner(isr)
while (sc.hasNextLine()) println(sc.nextLine())
sc.close()
}
Or simplier, since Kotlin 1.2
import java.net.URL
fun main(args: Array<String>){
println(URL("https://sourceforge.net").readText())
}
local(x = curl('https://sourceforge.net'))
local(y = #x->result)
#y->asString
If a site with an invalid SSL Cert is encountered the curl type throws the following error:
- Output:
FAILURE: 60 Peer certificate cannot be authenticated with given CA certificates
- Windows:
ch = xtra("Curl").new()
CURLOPT_URL = 10002
ch.setOption(CURLOPT_URL, "https://sourceforge.net")
res = ch.exec(1)
if integerP(res) then
put "Error:" && curl_error(res)
else
put "Result:" && res.readRawString(res.length)
end if
-- "Result: <!doctype html> ..."- Mac OS X:
sx = xtra("Shell").new()
put sx.shell_cmd("curl https://sourceforge.net")Blocking version
libURLSetSSLVerification true --check cert
get URL "https://sourceforge.net/"Non-blocking version, execute getWebResource
on myUrlDownloadFinished
get URL "https://sourceforge.net/" -- this will now fetch a locally cached copy
put it
end myUrlDownloadFinished
command getWebResource
libURLFollowHttpRedirects true
libURLSetSSLVerification true --check cert
load URL "https://sourceforge.net/" with message "myUrlDownloadFinished"
end getWebResourceVirtually identical to the HTTP Task.
To test it yourself; rez a box on the ground, and add the following as a New Script.
string sURL = "https://SourceForge.Net/";
key kHttpRequestId;
default {
state_entry() {
kHttpRequestId = llHTTPRequest(sURL, [], "");
}
http_response(key kRequestId, integer iStatus, list lMetaData, string sBody) {
if(kRequestId==kHttpRequestId) {
llOwnerSay("Status="+(string)iStatus);
integer x = 0;
for(x=0 ; x<llGetListLength(lMetaData) ; x++) {
llOwnerSay("llList2String(lMetaData, "+(string)x+")="+llList2String(lMetaData, x));
}
list lBody = llParseString2List(sBody, ["\n"], []);
for(x=0 ; x<llGetListLength(lBody) ; x++) {
llOwnerSay("llList2String(lBody, "+(string)x+")="+llList2String(lBody, x));
}
}
}
}
Output:
Status=200
llList2String(lMetaData, 0)=0
llList2String(lMetaData, 1)=2048
llList2String(lBody, 0)=<!doctype html>
llList2String(lBody, 1)=<!-- Server: sfs-consume-7 -->
llList2String(lBody, 2)=<!--[if lt IE 7 ]> <html lang="en" class="no-js ie6" > <![endif]-->
llList2String(lBody, 3)=<!--[if IE 7 ]> <html lang="en" class="no-js ie7" > <![endif]-->
llList2String(lBody, 4)=<!--[if IE 8 ]> <html lang="en" class="no-js ie8" > <![endif]-->
llList2String(lBody, 5)=<!--[if IE 9 ]> <html lang="en" class="no-js ie9" > <![endif]-->
llList2String(lBody, 6)=<!--[if (gt IE 9)|!(IE)]>--> <html lang="en" class="no-js"> <!--<![endif]-->
llList2String(lBody, 7)= <head>
llList2String(lBody, 8)= <meta charset="utf-8">
llList2String(lBody, 9)=
llList2String(lBody, 10)= <meta id="webtracker" name="webtracker" content='{"event_id": "ea71f064-ca28-11e1-98cc-0019b9f0e8fc"}'>
llList2String(lBody, 11)= <meta name="description" content="Free, secure and fast downloads from the largest Open Source applications and software directory - SourceForge.net">
llList2String(lBody, 12)= <meta name="keywords" content="Open Source, Open Source Software, Development, Community, Source Code, Secure, Downloads, Free Software">
llList2String(lBody, 13)=<meta name="msvalidate.01" content="0279349BB9CF7ACA882F86F29C50D3EA" />
llList2String(lBody, 14)= <meta name="viewport" content="width=device-width, initial-scale=1.0">
llList2String(lBody, 15)= <title>SourceForge - Download, Develop and Publish Free Open Source Software</title>
llList2String(lBody, 16)= <link rel="shortcut icon" href="http://a.fsdn.com/con/img/sftheme/favicon.ico">
... ... ... ... ... ... ... ... ... ... ... ... ... ...
local request = require('http.request')
local headers, stream = request.new_from_uri("https://sourceforge.net/"):go()
local body = stream:get_body_as_string()
local status = headers:get(':status')
io.write(string.format('Status: %d\nBody: %s\n', status, body)
HTTPS requests can be also done with the much smaller libraries like LuaSec or lua-requests, but it currently don't support redirects, which is why I used lua-http in this example.
We use Async read from Microsoft.XMLHTTP So we use Threads (duration is in millisecond) M2000 can use COM objects, using Declare, Method and With statements. Using With statement we can make objects properties like ReadyState as variables (some of them as read only)
Module CheckIt {
Declare xml "Microsoft.XMLHTTP"
const testUrl$ = "https://georgekarras.blogspot.com/"
With xml, "readyState" as ReadyState
Method xml "Open", "Get", testUrl$, True ' True means Async
Method xml "send"
\\ We set a thread to count time
k=0
Thread {
k++
} as TimeOut interval 100
\\ In main thread we can check ReadyState and Mouse button
Task.Main 100 {
Print ReadyState
If ReadyState=4 then exit
if k>20 then exit ' 20*100= 2 sec
if mouse then exit ' exit if mouse click
}
\\ So now we can read
if ReadyState=4 then {
With xml, "responseText" AS AA$
\\ break AA$ to lines
Document BB$=AA$
\\ using line breaks as CRLF
Report BB$
}
Declare xml Nothing
}
CheckIt
content := URL:-Get( "https://www.google.ca/" );
Straight forward "Import" task. More complicated secure web access can be done using J/Link; essentially a link to Java API.
content=Import["https://sourceforge.net", "HTML"]
s=urlread('https://sourceforge.net/')
This example is essentially identical to the HTTP task because the WebClient object can be used with http:, https:, ftp: and file: uri's.
using System;
using System.Console;
using System.Net;
using System.IO;
module HTTP
{
Main() : void
{
def wc = WebClient();
def myStream = wc.OpenRead(https://sourceforge.com);
def sr = StreamReader(myStream);
WriteLine(sr.ReadToEnd());
myStream.Close()
}
}
(! "curl https://sourceforge.net")
Compile with nim c -d:ssl httpsClient.nim:
import httpclient
var client = newHttpClient()
echo client.getContent("https://sourceforge.net")
use HTTP;
class HttpsTest {
function : Main(args : String[]) ~ Nil {
client := HttpsClient->New();
lines := client->Get("https://sourceforge.net");
each(i : lines) {
lines->Get(i)->As(String)->PrintLine();
};
}
}(import (lib curl))
(define curl (make-curl))
(curl 'url "https://www.w3.org/")
(curl 'perform)
Using fphttpclient
{$mode objfpc}{$H+}
uses fphttpclient;
var
s: string;
hc: tfphttpclient;
begin
hc := tfphttpclient.create(nil);
try
s := hc.get('https://www.example.com')
finally
hc.free
end;
writeln(s)
end.
##
uses System.Net;
var wc := new WebClient();
var content := wc.DownloadString('https://example.com');
content.Println
use strict;
use LWP::UserAgent;
my $url = 'https://www.rosettacode.org';
my $response = LWP::UserAgent->new->get( $url );
$response->is_success or die "Failed to GET '$url': ", $response->status_line;
print $response->as_string;
Exactly the same as the HTTP#Phix task.
without js include builtins\libcurl.e curl_global_init() atom curl = curl_easy_init() curl_easy_setopt(curl, CURLOPT_URL, "https://sourceforge.net/") object res = curl_easy_perform_ex(curl) curl_easy_cleanup(curl) curl_global_cleanup() puts(1,res)
echo file_get_contents('https://sourceforge.net');
PicoLisp has no functionality for communicating with a HTTPS server (only for the other direction), but it is easy to use an external tool
(in '(curl "https://sourceforge.net") # Open a pipe to 'curl'
(out NIL (echo)) ) # Echo to standard outputint main() {
write("%s\n", Protocols.HTTP.get_url_data("https://sourceforge.net"));
}
Invoke-WebRequest 'https://www.rosettacode.org'
$wc = New-Object Net.WebClient
$wc.DownloadString('https://sourceforge.net')
If the certificate could not be validated (untrusted, self-signed, expired), then an Exception is thrown with the message “The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.” so certificate validation is done automatically by the method.
Python's urllib.request library, has support for SSL if the interpreter's underlying httplib libraries were compiled with SSL support. By default this will be the enabled for default Python installations on most platforms.
import urllib.request
print(urllib.request.urlopen("https://sourceforge.net/").read())
The basic idea is to use getURL (as with HTTP_Request), but with some extra parameters.
library(RCurl)
webpage <- getURL("https://sourceforge.net/", .opts=list(followlocation=TRUE, ssl.verifyhost=FALSE, ssl.verifypeer=FALSE))
In this case, the webpage output contains unprocessed characters, e.g. \" instead of " and \\ instead of \, so we need to process the markup.
wp <- readLines(tc <- textConnection(webpage))
close(tc)
Finally, we parse the HTML and find the interesting bit.
pagetree <- htmlTreeParse(wp)
pagetree$children$html
#lang racket
(require net/url)
(copy-port (get-pure-port (string->url "https://www.google.com")
#:redirections 100)
(current-output-port))
(formerly Perl 6)
There are several modules that provide HTTPS capability. WWW and HTTP::UserAgent are probably the most popular right now, but others exist.
use WWW;
say get 'https://sourceforge.net/';
or
use HTTP::UserAgent;
say HTTP::UserAgent.new.get('https://sourceforge.net/').content;
REALBasic provides an HTTPSecureSocket class for handling HTTPS connections. The 'Get' method of the HTTPSecureSocket is overloaded and can download data to a file or return data as a string, in both cases an optional timeout argument can be passed.
Dim sock As New HTTPSecureSocket
Print(sock.Get("https://sourceforge.net", 10)) //set the timeout period to 10 seconds.print read https://www.w3.org
cStr= download("http://sourceforge.net/")
see cStr + nlSee HTTP#RLaB
This solution doesn't use the open-uri convenience package that the HTTP Request#Ruby solution uses: the Net::HTTP object must be told to use SSL before the session is started.
require 'net/https'
require 'uri'
require 'pp'
uri = URI.parse('https://sourceforge.net')
http = Net::HTTP.new(uri.host,uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
http.start do
content = http.get(uri)
p [content.code, content.message]
pp content.to_hash
puts content.body
end
outputs
["302", "Found"]
{"location"=>["http://sourceforge.net/"],
"content-type"=>["text/html; charset=UTF-8"],
"connection"=>["close"],
"server"=>["nginx/0.7.60"],
"date"=>["Sun, 30 Aug 2009 20:20:07 GMT"],
"content-length"=>["229"],
"set-cookie"=>
["sf.consume=89f65c6fadd222338b2f3de6f8e8a17b2c8f67c2gAJ9cQEoVQhfZXhwaXJlc3ECY2RhdGV0aW1lCmRhdGV0aW1lCnEDVQoH9gETAw4HAAAAhVJxBFUDX2lkcQVVIDEyOWI2MmVkOWMwMWYxYWZiYzE5Y2JhYzcwZDMxYTE4cQZVDl9hY2Nlc3NlZF90aW1lcQdHQdKmt73UN21VDl9jcmVhdGlvbl90aW1lcQhHQdKmt73UN2V1Lg==; expires=Tue, 19-Jan-2038 03:14:07 GMT; Path=/"]}
<html>
<head>
<title>302 Found</title>
</head>
<body>
<h1>302 Found</h1>
The resource was found at <a href="http://sourceforge.net/">http://sourceforge.net/</a>;
you should be redirected automatically.
</body>
</html>
extern crate reqwest;
fn main() {
let response = match reqwest::blocking::get("https://sourceforge.net") {
Ok(response) => response,
Err(e) => panic!("error encountered while making request: {:?}", e),
};
println!("{}", response.text().unwrap());
}
- Output:
<!-- Server: sfs-consume-7 -->
<html class="no-js" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no" />
<script>
var __gdpr = true;
var __ccpa = false;
...
import scala.io.Source
object HttpsTest extends App {
System.setProperty("http.agent", "*")
Source.fromURL("https://sourceforge.net").getLines.foreach(println)
}
The library gethttps.s7i defines the function getHttps which uses the HTTPS protocol go get a file.
$ include "seed7_05.s7i";
include "gethttps.s7i";
include "utf8.s7i";
const proc: main is func
begin
writeln(STD_UTF8_OUT, getHttps("sourceforge.net"));
end func;require('LWP::UserAgent')
require('LWP::Protocol::https')
func get(url) {
static ua = %O<LWP::UserAgent>.new(
agent => 'Mozilla/5.0',
ssl_opts => Hash(verify_hostname => 1),
)
var resp = ua.get(url)
if (resp.is_success) {
return resp.decoded_content
}
die "Failed to GET #{url}: #{resp.status_line}"
}
say get("https://rosettacode.org")
import Foundation
// With https
let request = NSURLRequest(URL: NSURL(string: "https://sourceforge.net")!)
NSURLConnection.sendAsynchronousRequest(request, queue: NSOperationQueue()) {res, data, err in // callback
// data is binary
if (data != nil) {
let string = NSString(data: data!, encoding: NSUTF8StringEncoding)
println(string)
}
}
CFRunLoopRun() // dispatch
The preliminary function from the standard library actually pipes a shell invocation of 'curl', thus it might not work in the playground, as shell invocation is there inhibited.
main (parms):+
url =: 'https://www.rosettacode.org'
?# lne =: network url url give lines
print lne- Output:
html> <head><title>301 Moved Permanently</title></head> <body> <center><h1>301 Moved Permanently</h1></center> <hr><center>nginx</center> </body> </html>
Though Tcl's built-in http package does not understand SSL, it does support the registration of external handlers to accommodate additional protocols. This allows the use of the Tls package to supply the missing functionality with only a single line to complete the registration.
package require http
package require tls
# Tell the http package what to do with “https:” URLs.
#
# First argument is the protocol name, second the default port, and
# third the connection builder command
http::register "https" 443 ::tls::socket
# Make a secure connection, which is almost identical to normal
# connections except for the different protocol in the URL.
set token [http::geturl "https://sourceforge.net/"]
# Now as for conventional use of the “http” package
puts [http::data $token]
http::cleanup $token
$$ MODE TUSCRIPT
SET DATEN = REQUEST ("https://sourceforge.net")
*{daten}curl -k -s -L https://sourceforge.net/
Based on code at How to retrieve HTML web pages with VBScript via the Microsoft.XmlHttp object
Option Explicit
Const sURL="https://sourceforge.net/"
Dim oHTTP
Set oHTTP = CreateObject("Microsoft.XmlHTTP")
On Error Resume Next
oHTTP.Open "GET", sURL, False
oHTTP.Send ""
If Err.Number = 0 Then
WScript.Echo oHTTP.responseText
Else
Wscript.Echo "error " & Err.Number & ": " & Err.Description
End If
Set oHTTP = Nothing
Sub Main()
Dim HttpReq As WinHttp.WinHttpRequest
' in the "references" dialog of the IDE, check
' "Microsoft WinHTTP Services, version 5.1" (winhttp.dll)
Const HTTPREQUEST_PROXYSETTING_PROXY As Long = 2
Const WINHTTP_FLAG_SECURE_PROTOCOL_TLS1 As Long = &H80&
Const WINHTTP_FLAG_SECURE_PROTOCOL_TLS1_1 As Long = &H200&
Const WINHTTP_FLAG_SECURE_PROTOCOL_TLS1_2 As Long = &H800&
#Const USE_PROXY = 1
Set HttpReq = New WinHttp.WinHttpRequest
HttpReq.Open "GET", "https://groups.google.com/robots.txt"
HttpReq.Option(WinHttpRequestOption_SecureProtocols) = WINHTTP_FLAG_SECURE_PROTOCOL_TLS1 Or _
WINHTTP_FLAG_SECURE_PROTOCOL_TLS1_1 Or _
WINHTTP_FLAG_SECURE_PROTOCOL_TLS1_2
#If USE_PROXY Then
HttpReq.SetProxy HTTPREQUEST_PROXYSETTING_PROXY, "my_proxy:80"
#End If
HttpReq.SetTimeouts 1000, 1000, 1000, 1000
HttpReq.Send
Debug.Print HttpReq.ResponseText
End Sub
Imports System.Net
Dim client As WebClient = New WebClient()
Dim content As String = client.DownloadString("https://sourceforge.net")
Console.WriteLine(content)
This assumes the 'curl' command line utility is available on the system.
Uses the 'os' sub-module of Wren-std.
import "os" for Process
Process.exec("curl -s -L https://www.w3.org/")
NASM 2.16
;; The name of this wont get confusing :]
%macro prolog 1
push rbp
mov rbp, rsp
sub rsp, %1
%endmacro
%macro epilog 1
add rsp, %1
pop rbp
%endmacro
;;default rel does some crazy shit
;;so here we are because of it. T_T
%macro xcall 1
call [rel %1 wrt ..got]
%endmacro
%macro xlea 2
lea %1, [rel %2]
%endmacro
extern curl_easy_init
extern curl_easy_setopt
extern curl_easy_perform
extern curl_easy_cleanup
extern printf
%define CURLOPT_URL 0x2712
%define CURLOPT_FOLLOWLOCATION 0x34
%define CURLOPT_ERRORBUFFER 0x271A
%define CURLE_OK 0
section .rodata
szformat db "%s",10,0
sznoargs db "usage:",10," curl <url>",10,0
section .bss
buff resb 1024
section .text
global main
main:
prolog 4*8
%define url [rbp-1*8]
%define curl [rbp-2*8]
cmp edi, 1
jle __main_noargs
mov rax, qword [rsi+1*8]
mov qword url, rax
xcall curl_easy_init
mov qword curl, rax
cmp rax, 0
je __main_exit
mov rdi, qword curl
mov rdx, qword url
mov esi, CURLOPT_URL
xcall curl_easy_setopt
mov rdi, qword curl
mov esi, CURLOPT_FOLLOWLOCATION
mov edx, 1
xcall curl_easy_setopt
mov rdi, qword curl
xlea rdx, buff
mov esi, CURLOPT_ERRORBUFFER
xcall curl_easy_setopt
mov rdi, qword curl
xcall curl_easy_perform
cmp eax, 0
je __main_exit
xlea rdi, szformat
xlea rsi, buff
xcall printf
jmp __main_exit
__main_noargs:
xlea rdi, sznoargs
xcall printf
__main_exit:
mov rdi, qword curl
xcall curl_easy_cleanup
epilog 4*8
ret
This uses OpenSSL's libraries(libssl[-lssl] libcrypto[-lcrypto]) BIO_* functions to make the connection, send and recv data. There are a multitude of ways to use OpenSSL to achieve a TLS connection, this just being one.
%macro prolog 1
push rbp
mov rbp, rsp
sub rsp, %1
%endmacro
%macro epilog 1
add rsp, %1
pop rbp
%endmacro
%macro xcall 1
call [rel %1 wrt ..got]
%endmacro
%macro xlea 2
lea %1, [rel %2]
%endmacro
extern printf
extern strlen
extern strdup
extern strchr
extern free
extern exit
;; --> OpenSSL Stuffs <--
extern BIO_new_ssl_connect
extern BIO_ctrl
extern BIO_write
extern BIO_read
extern BIO_free_all
extern TLS_client_method
extern SSL_CTX_new
extern SSL_CTX_free
extern SSL_CTX_set_verify
extern SSL_CTX_set_default_verify_paths
extern SSL_ctrl
%define BIO_C_SET_CONNECT 100
%define BIO_C_DO_STATE 101
%define BIO_C_GET_SSL 110
%define SSL_CTRL_SET_TLSHOST 55
%define SSL_VERIFY_PEER 1
section .text
global xrequest
xrequest:
prolog 6*8
%define _req_host rbp-1*8
%define _req_pack rbp-2*8
%define _req_buff rbp-3*8
%define _req_ssl rbp-4*8
%define _req_ctx rbp-5*8
%define _req_bio rbp-6*8
mov rax, rdi
mov qword [_req_host], rax
mov rax, rsi
mov qword [_req_pack], rax
mov rax, rdx
mov qword [_req_buff], rax
xcall TLS_client_method
mov rdi, rax
xcall SSL_CTX_new
test rax, rax
jz __req_err_ctx
mov qword [_req_ctx], rax
mov rdi, qword [_req_ctx]
xcall SSL_CTX_set_default_verify_paths
test eax, eax
jz __req_err_verify
mov rdx, 0
mov rsi, SSL_VERIFY_PEER
mov rdi, qword [_req_ctx]
xcall SSL_CTX_set_verify
;;-->> Create Bio chain
mov rdi, qword [_req_ctx]
xcall BIO_new_ssl_connect
test rax, rax
jz __req_err_bio
mov qword [_req_bio], rax
;;-->> Get SSL object
lea rcx, [_req_ssl]
mov rdx, 0
mov rsi, BIO_C_GET_SSL
mov rdi, qword [_req_bio]
xcall BIO_ctrl
;; -->> Set the SNI hostname.
mov rdi, qword [_req_host]
xcall strdup
mov rdx, rax ; rdx now holds the copy
push rdx ; Push pointer to stack
mov rdi, rdx
mov rsi, ':'
xcall strchr
test rax, rax
jz __req_sni_set
mov byte [rax], 0 ; Null-terminate at colon
__req_sni_set:
mov rdi, qword [_req_ssl]
mov rsi, SSL_CTRL_SET_TLSHOST
mov rdx, 0
mov rcx, [rsp]
xcall SSL_ctrl
pop rdi
xcall free
;; -->>Set Hostname (BIO expects "host:port")
mov rdi, qword [_req_bio]
mov rsi, BIO_C_SET_CONNECT
mov rdx, 0
mov rcx, qword [_req_host]
xcall BIO_ctrl
mov rdi, qword [_req_bio]
mov rsi, BIO_C_DO_STATE
mov rdx, 0
mov rcx, 0
xcall BIO_ctrl
cmp rax, 1
jl __req_err_conn
;; -->> Send Request
mov rdi, qword [_req_pack]
xcall strlen
mov rdx, rax
mov rdi, qword [_req_bio]
mov rsi, qword [_req_pack]
xcall BIO_write
; -->> Read Response Loop
__req_read_loop:
mov rdi, qword [_req_bio]
mov rsi, qword [_req_buff]
mov rdx, 1024
xcall BIO_read
test eax, eax
jle __req_exit
add qword [_req_buff], rax
jmp __req_read_loop
__req_err_bio:
mov r13, -1
jmp __req_exit
__req_err_verify:
mov r13, -2
jmp __req_exit
__req_err_ctx:
;;-->> CTX Error
mov r13, -3
jmp __req_exit
__req_err_conn:
mov r13, -4
;; fall through..
__req_exit:
mov r13, rax
mov rdi, qword [_req_bio]
xcall BIO_free_all
mov rdi, qword [_req_ctx]
xcall SSL_CTX_free
mov rax, r13
epilog 6*8
ret
global main
main:
prolog 4*8
; request(hostname, request_str, buffer)
xlea rdi, hostname
xlea rsi, request_str
xlea rdx, buffer
call xrequest
mov rsi, rax
xlea rdi, fmt_ret
xcall printf
; Print the response buffer
xlea rdi, fmt_str
xlea rsi, buffer
xor rax, rax
xcall printf
xor rdi, rdi
xcall exit
prolog 4*8 ;;Never gets here but w/e.
ret
section .data
hostname db "google.com:443", 0
request_str db "GET / HTTP/1.1", 13, 10
db "Host: google.com", 13, 10
db "Connection: close", 13, 10
db "User-Agent: TLS-TEST/0.5", 13, 10
db 13, 10, 0
fmt_str db "%s", 0
fmt_ret db "%i",10,0
section .bss
buffer resb 16384
!ys-0
defn main(url='https://www.w3.org/'):
print: http/get(url).body
- Output:
$ ys https.ys
<!doctype html>
<html dir="ltr" lang="en" class="no-js">
<head>
<title>W3C</title>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="home" href="https://www.w3.org/"/>
<link rel="icon" type="image/png" href="https://www.w3.org/assets/logos/w3c-2025/favicons/favicon-32.png"/>
<link rel="apple-touch-icon" type="image/png" href="https://www.w3.org/assets/logos/w3c-2025/favicons/favicon-180.png"/>
<link rel="stylesheet" href="https://www.w3.org/assets/website-2021/styles/core.css?ver=1.5" media="screen"/>
<!--
CSS Mustard Cut
Print (Edge doesn't apply to print otherwise)
Edge, Chrome 39+, Opera 26+, Safari 9+, iOS 9+, Android ~5+, Android UCBrowser ~11.8+
FF 47+
-->
<link rel="stylesheet" id="advanced-stylesheet"
href="https://www.w3.org/assets/website-2021/styles/advanced.css?ver=1.5" media="
only print,
only all and (pointer: fine), only all and (pointer: coarse), only all and (pointer: none),
only all and (min--moz-device-pixel-ratio:0) and (display-mode:browser), (min--moz-device-pixel-ratio:0) and (display-mode:fullscreen)
">
<link rel="stylesheet" href="https://www.w3.org/assets/website-2021/styles/print.css" media="print"/>
<script src="https://www.w3.org/assets/website-2021/js/libraries/fontfaceobserver.js"></script>
<script>
var myFont = new FontFaceObserver('Noto Sans');
Promise.all([myFont.load()]).then(function () {
document.documentElement.className += " fonts-loaded";
});
(function () {
var linkEl = document.getElementById('advanced-stylesheet');
if (window.matchMedia && window.matchMedia(linkEl.media).matches) {
var script = document.createElement('script');
script.src = 'https://www.w3.org/assets/website-2021/js/main.js?ver=1.4';
script.defer = true;
document.querySelector('head').appendChild(script);
(function (H) {
H.className = H.className.replace(/\bno-js\b/, 'js')
})(document.documentElement);
}
})();
</script>
Using the cURL library to do the heavy lifting:
zkl: var ZC=Import("zklCurl")
zkl: var data=ZC().get("https://sourceforge.net")
L(Data(36,265),826,0)get returns the text of the response along with two counts: the bytes of header in front of the html code and the byte count of stuff after the end of the page. So, if you wanted to look at the header:
zkl: data[0][0,data[1]).text HTTP/1.1 200 OK Server: nginx Date: Sun, 23 Mar 2014 07:36:51 GMT Content-Type: text/html; charset=utf-8 Connection: close ...
or some of the html:
zkl: data[0][data[1],200).text <!doctype html> <!-- Server: sfs-consume-8 --> <!--[if lt IE 7 ]> <html lang="en" class="no-js ie6"> <![endif]--> <!--[if IE 7 ]> <html lang="en" class="no-js ie7"> <![endif]--> <!--[if IE 8 ]>