ObjFW  History of src/OFHTTPClient.m of 4320b1bf7dfc515ef712636cfbbc7d00fbfa4dd4dacd606184d20d136f1a2590

This deprecates OFHTTPRequestMethodName(), which returns a C string.
APIs should avoid C strings as much as possible.

This function was initially only used internally, where this was fine.
However, when it was made public, it should have been converted to
OFString at the same time.

Adds OFHTTPRequestMethodParseString() for consistency, which behaves the
same as OFHTTPRequestMethodParseName() and deprecates it. file: [771a949bd6] check-in: [12c09ef41e] user: js, branch: trunk, size: 30943 [annotate] [blame] [check-ins using] [diff]

This allows for a much cleaner solution to avoid the internal read
buffer of e.g. a TLS connection never being processed while waiting for
a delimiter. file: [0348d66062] check-in: [a61e0594b4] user: js, branch: trunk, size: 30941 [annotate] [blame] [check-ins using] [diff]

This made reading the response synchronously fail. file: [0a2253ded6] check-in: [3a8aa96e16] user: js, branch: trunk, size: 31216 [annotate] [blame] [check-ins using] [diff]

The previous API could never work cleanly and would always require
hacks, as it needed intercepting all interactions of OFTCPSocket with
the raw socket and did not work at all if the OFTCPSocket had anything
in its read buffer before starting the TLS handshake. This also could
not be fixed easily, as it would have required the object to contain two
read buffers, one for the unencrypted connection and one for the
encrypted connection. There was also no clean way to perform the
handshake in a non-blocking way.

The new API is a lot cleaner and requires none of the hacks, but using
it requires slightly more work. But this is more than made up for by
making a fully asynchronous handshake possible. It uses the concept of a
stream wrapping another stream, meaning the entire connecting part is
being handled by OFTCPSocket and then the connected socket is passed off
to OFTLSStream to wrap it. This also makes for a lot cleaner separation
of concerns. file: [0ef7b9c0c4] check-in: [d30efa8bbf] user: js, branch: trunk, size: 30982 [annotate] [blame] [check-ins using] [diff]

The old write API made it too easy to lose bytes when a stream is set to
non-blocking mode. The new API always throws when not all bytes were
written, which forces handling the number of bytes being written being
smaller than the number of bytes requested to be written. file: [63931bf44e] check-in: [d1d36ae522] user: js, branch: trunk, size: 29819 [annotate] [blame] [check-ins using] [diff]

This should make it easier to add TLS support using various
implementations. file: [95bd8d7b45] check-in: [34cb121dc5] user: js, branch: trunk, size: 30051 [annotate] [blame] [check-ins using] [diff]

This is only migrating some places, others will be migrated as they are
touched. file: [4079ebb727] check-in: [a0fd103a0b] user: js, branch: trunk, size: 30144 [annotate] [blame] [check-ins using] [diff]

The new names should be more accurate. file: [5174ef54a9] check-in: [627511b032] user: js, branch: trunk, size: 30271 [annotate] [blame] [check-ins using] [diff]

-[allocMemoryWithSize:] has book keeping overhead that is unnecessary
for temporary memory. file: [72f7071464] check-in: [0ddaac3f5b] user: js, branch: trunk, size: 30272 [annotate] [blame] [check-ins using] [diff]

While it makes sense to wrap malloc and calloc to replace the error
checking with exceptions, it does not make sense to wrap free. file: [eedb53c084] check-in: [3d8286feee] user: js, branch: trunk, size: 30272 [annotate] [blame] [check-ins using] [diff]

Instead, add an exception to didPerformRequest, to make it more
consistent with other places. file: [62848f22ab] check-in: [7f102e8c95] user: js, branch: trunk, size: 30336 [annotate] [blame] [check-ins using] [diff]

It used to be a short in some places and an int in other places. file: [f132e311be] check-in: [6e42ee482f] user: js, branch: trunk, size: 30374 [annotate] [blame] [check-ins using] [diff]

Since C guarantees minimum sizes for char, short, int, long and long
long, these can be used instead. file: [561ec71175] check-in: [25c985fec1] user: js, branch: trunk, size: 30364 [annotate] [blame] [check-ins using] [diff]

This solves the old signed vs. unsigned problem and allows for more
bases than just 8, 10 and 16, as well as auto-detection of the base (if
base is 0). file: [d541eab337] check-in: [b6ee372b98] user: js, branch: trunk, size: 30357 [annotate] [blame] [check-ins using] [diff]

This is a new superclass for OFTCPSocket to allow sharing code with
other IP stream socket types, namely SCTP. file: [39a2892f3d] check-in: [3dfe642dd3] user: js, branch: trunk, size: 30125 [annotate] [blame] [check-ins using] [diff]

This means refusing to close twice, calling -[close] from -[dealloc] and
not calling -[cancelAsyncRequests].

Calling -[cancelAsyncRequests] in -[close] is too dangerous, as -[close]
gets called by -[dealloc]: If the queue is the last reference to the
object, at the point where -[cancelAsyncRequests] removes it from the
queue, the object will start to deallocate and call into
-[cancelAsyncRequests] again, which is still in the middle of removing
it and now finds itself with an inconsistent state. file: [cc1af7fa6d] check-in: [3b43d51006] user: js, branch: trunk, size: 28008 [annotate] [blame] [check-ins using] [diff]

When there's no keep-alive and the socket is not kept, it fails when the
OFHTTPClientResponse gets removed from the kernel event observer. file: [90f1585c87] check-in: [b4810363ea] user: js, branch: trunk, size: 27846 [annotate] [blame] [check-ins using] [diff]

-[OFURL port] used to be an integer, but was changed to being an
OFNumber, so needs -[isEqual:] now. file: [e2bb72d5a2] check-in: [eadd1a0c54] user: js, branch: trunk, size: 27939 [annotate] [blame] [check-ins using] [diff]

It was not very useful as for any OFHTTPClient there can only be one
active request. file: [830dad71cc] check-in: [eaf458c1e6] user: js, branch: trunk, size: 28132 [annotate] [blame] [check-ins using] [diff]

This avoids the entire problem of keeping the buffer alive until the
write finished. file: [c6a81ba86b] check-in: [f218986f51] user: js, branch: trunk, size: 29505 [annotate] [blame] [check-ins using] [diff]

Otherwise, there would be two methods for every operation: One for
success and one for failure. It also makes it easy to forget about
handling failure, so it's better to always pass an optional exception. file: [209c55758d] check-in: [064dbe5127] user: js, branch: trunk, size: 29553 [annotate] [blame] [check-ins using] [diff]

The target / selector approach had several drawbacks:

* It was inconvenient to use, as for every read or write, a target,
selector and context would need to be specified.
* It lacked any kind of type-safety and would not even warn about using
a callback method with the wrong number of parameters.
* It encouraged using a different callback method for each read or
write call, which results in code that is hard to follow and also
slower (as it needs to recreate the async operation with a new
callback every time). file: [376bac39d6] check-in: [d16ad96cbd] user: js, branch: trunk, size: 29459 [annotate] [blame] [check-ins using] [diff]

This makes it more clear that this has nothing to do with the body of
the response, as the body of the response is read from the
OFHTTPResponse itself. file: [325ef4384d] check-in: [d375061886] user: js, branch: trunk, size: 30627 [annotate] [blame] [check-ins using] [diff]

This uses -[asyncPerformRequest:redirects:context:] under the hood and
runs a runloop until it finished. file: [0d10bb98e9] check-in: [f80b0d270c] user: js, branch: trunk, size: 31128 [annotate] [blame] [check-ins using] [diff]

This is in preparation for removing the body from OFHTTPRequest.
Having it as OFData that is part of the OFHTTPRequest was a bad idea, as
it does not allow streaming. file: [aea3f9620c] check-in: [8681bba25e] user: js, branch: trunk, size: 26207 [annotate] [blame] [check-ins using] [diff]

Instead of letting OFStream conform to it, which itself does not really
conform to it, let all the subclasses that actually do conform to it. file: [a6980a9e89] check-in: [69749b6a5b] user: js, branch: trunk, size: 23984 [annotate] [blame] [check-ins using] [diff]

This now URL-encodes everything before sending the request. This wasn't
necessary before, as OFURL contained the encoded variant in its
properties. Since it now contains the unencoded variant, we need to
encode it before sending the request. file: [69a4197ea4] check-in: [35b398fae4] user: js, branch: trunk, size: 23921 [annotate] [blame] [check-ins using] [diff]

This removes the special handling dependent on the scheme. file: [b30a887ade] check-in: [0f995db06d] user: js, branch: trunk, size: 23861 [annotate] [blame] [check-ins using] [diff]

Instead, explicitly declare them, as OF_ASSUME_NONNULL_{BEGIN,END} does
not apply to implicit return types. This means that after this commit,
all init methods have a nonnull return type, as they should have. file: [901849c7c7] check-in: [2f4e0df8be] user: js, branch: trunk, size: 23833 [annotate] [blame] [check-ins using] [diff]

This makes it clear that the request is handled asynchronously and makes
it possible to reintroduce a synchronous version later. file: [3d75f15cfb] check-in: [0319fe1eb9] user: js, branch: trunk, size: 23033 [annotate] [blame] [check-ins using] [diff]

This does not make OFHTTPClientResponse completely asynchronous yet.

Since -[initWithURL:] does not work well with an asynchronous API, it no
longer supports HTTP URLs. However, a new, asynchronous API will be
added as a replacement later on. file: [fb6dd8b062] check-in: [56a18442e2] user: js, branch: trunk, size: 21788 [annotate] [blame] [check-ins using] [diff]

This is in preparation for making OFHTTPClient asynchronous. file: [aa3131b9d1] check-in: [d3d4d34dad] user: js, branch: trunk, size: 18637 [annotate] [blame] [check-ins using] [diff]

Instead, let reads return 0 and let writes append after the end. file: [aa5274f753] check-in: [f9cd4f9cab] user: js, branch: trunk, size: 17787 [annotate] [blame] [check-ins using] [diff]

This matches Apple's style. file: [530dbe6279] check-in: [6b77a5dd8b] user: js, branch: trunk, size: 17739 [annotate] [blame] [check-ins using] [diff]

Casts are now written like types in variable declarations. file: [8ee3bb44d0] check-in: [4af49a13c3] user: js, branch: trunk, size: 17745 [annotate] [blame] [check-ins using] [diff]

Forgot to add 2017, even though I already did quite some changes in
2017. file: [9bb4558e66] check-in: [44f45c2e35] user: js, branch: trunk, size: 17591 [annotate] [blame] [check-ins using] [diff]

These are independent of the locale and work on the ASCII character set.

Unlike the C ones, these are 8-bit safe, meaning if a character > 0x7F
is passed, is{alpha,alnum} returns false and to{upper,lower} returns the
original character. file: [f4eacb56f1] check-in: [d9eb7b50b3] user: js, branch: trunk, size: 17585 [annotate] [blame] [check-ins using] [diff]

This allows ofhttp to display all headers for the redirect in verbose
mode (the change for this is included). file: [9433d07795] check-in: [a509ab7e91] user: js, branch: trunk, size: 17546 [annotate] [blame] [check-ins using] [diff]

URL decoding and reencoding is not lossless: For example, if the query
was foo=bar&qux=foo%25bar, it will be decoded to foo=bar&qux=foo&bar and
then reencoded to foo=bar%25qux=foo%25bar, which is a different thing.

The only way to solve this is to let the application handle the URL
decoding and encoding according to its own rules, as those might be
different depending on the application. file: [5d3409efa0] check-in: [516517deb3] user: js, branch: trunk, size: 17510 [annotate] [blame] [check-ins using] [diff]

The Host was still pointing to the old Host on a redirect to a different
host. file: [01bffed1c2] check-in: [412c005e2b] user: js, branch: trunk, size: 17585 [annotate] [blame] [check-ins using] [diff]

OFURL used to return all URL parts escaped, however, when this was
changed, OFHTTPClient was not adjusted. file: [c92bc66cbb] check-in: [c25601d462] user: js, branch: trunk, size: 17583 [annotate] [blame] [check-ins using] [diff]

Right now, this is only useful for OFTCPSocket, as this is the only
class so far not establishing the stream in the init method. However,
this adds the general infrastructure to allow reuse to all subclasses of
OFStream. file: [2ea850eecb] check-in: [1de551cb5f] user: js, branch: trunk, size: 17499 [annotate] [blame] [check-ins using] [diff]

This is required as passing something signed to it is an error on
NetBSD, but passing anything else than a char is undefined on Linux. file: [cf407bada1] check-in: [09a0d12bae] user: js, branch: trunk, size: 17482 [annotate] [blame] [check-ins using] [diff]

This makes it possible for the user to override Authorization and
Connection. file: [c0960d74b1] check-in: [b9c811a236] user: js, branch: trunk, size: 17275 [annotate] [blame] [check-ins using] [diff]

This should randomize the order of all headers now, preventing
fingerprinting by the order of HTTP headers. file: [ad2a18a279] check-in: [58d4025602] user: js, branch: trunk, size: 17164 [annotate] [blame] [check-ins using] [diff]

While at it, also update the mail address. file: [9d203aa5e2] check-in: [cec0f072f8] user: js, branch: 0.8, size: 16946 [annotate] [blame] [check-ins using] [diff]

While at it, also update the mail address. file: [f0a87a8790] check-in: [2a27cf3000] user: js, branch: trunk, size: 16733 [annotate] [blame] [check-ins using] [diff]

This increases the required GCC version from 4.0 to 4.6 (exception:
Apple GCC, which already supports this with >= 4.0 starting with OS X
10.5). Since even GCC 4.6 is really old by now, there is no point in
still supporting something even older and making the code ugly because
of that. While some hardware and OS support was dropped from GCC 4.6
compared to GCC 4.0, there is nothing in there that would be an
interesting target with the exception of BeOS maybe - but a port to BeOS
can also be achieved using the Haiku support. The other dropped OSes are
mostly old versions of OSes while newer ones are still being supported
(and those newer versions of those OSes still support the same
hardware). file: [9951bb361f] check-in: [48980f2297] user: js, branch: trunk, size: 16729 [annotate] [blame] [check-ins using] [diff]

The old behaviour where the exception would access errno directly on
creation of the exception was very fragile. The two main problems with
it were that sometimes it would pick up an errno even though none had
been set and in other cases that when the exception was created errno
had already been overridden.

This also greatly increases errno handling on Win32, especially in
conjunction with sockets. It can still be improved further, though. file: [24ccd00731] check-in: [62e2de30b9] user: js, branch: trunk, size: 16153 [annotate] [blame] [check-ins using] [diff]

* Correctly handle inclusion of port in Host: for https
* Better name for OF_createSocketForRequest: file: [ba8944b933] check-in: [4b8c4cac59] user: js, branch: trunk, size: 16192 [annotate] [blame] [check-ins using] [diff]

While Cocoa includes it, RFC 1738 explicitly states that the "/" between
host / port and path is not part of the path. file: [ab4c72054a] check-in: [d2487bc7e1] user: js, branch: trunk, size: 16015 [annotate] [blame] [check-ins using] [diff]

In case the entity body has been set, but the headers don't contain
Content-Type and/or Content-Length, assume a default of
"application/x-www-form-urlencoded; charset=UTF-8" for the Content-Type
and set the Content-Length to the number of bytes of the entity body. file: [d38e8f0fed] check-in: [75c12eaf7b] user: js, branch: trunk, size: 16064 [annotate] [blame] [check-ins using] [diff]

This means that OFObject.h imports macros.h now, making it unnecessary
to manually import macros.h in almost every file. And while at it, also
import autorelease.h in OFObject.h, so that this doesn't need to be
manually imported in almost every file as well. file: [bf3f67d78a] check-in: [13ee56edf3] user: js, branch: trunk, size: 15180 [annotate] [blame] [check-ins using] [diff]

This replaces -[numberOfBytesInReadBuffer], as it's not always known how
many bytes there are exactly and thus the number would often be wrong
(e.g. because data is compressed). file: [26d3992686] check-in: [acc999a75e] user: js, branch: trunk, size: 15224 [annotate] [blame] [check-ins using] [diff]

This is in preparation for adding UDP sockets, as UDP sockets and TCP
sockets have no common superclass, as one is stream-oriented while the
other is packet-oriented.

Read and write exceptions are for any object now, as they are useful for
a lot more than just for streams, while the others (bind, listen, etc.)
are for any socket now (the type is id in this case, though, as there is
no common superclass). file: [3007afbd2c] check-in: [8d2a5052fd] user: js, branch: trunk, size: 15246 [annotate] [blame] [check-ins using] [diff]

A file documenting the coding style will be written soon. This will
hopefully prevent conflicts in the future, such as whether static
functions are written in camelCase or_with_underscores, like was the
case here. file: [80b58be37f] check-in: [fa6496efc7] user: js, branch: trunk, size: 15240 [annotate] [blame] [check-ins using] [diff]

-[OFHTTPClient close] now closes the connection, no matter whether a
reply still exists. -[OFHTTPRequestReply close] only disposes of the
reference to the socket now, so that if the OFHTTPClient does not exist
anymore the socket is closed, but not if the OFHTTPClient is still alive
and in keep-alive mode. This is important so that closing a single reply
does not close the client even though keep-alive was used. file: [65175ae100] check-in: [6ad892733b] user: js, branch: trunk, size: 14602 [annotate] [blame] [check-ins using] [diff]

This mostly removes the argument for the class in which the exception
occurred. As backtraces were recently added for all platforms, the
passed class does not give any extra information on where the exception
occurred anymore.

This also removes a few other arguments which were not too helpful. In
the past, the idea was to pass as many arguments as possible so that it
is easier to find the origin of the exception. However, as backtraces
are a much better way to find the origin, those are not useful anymore
and just make the exception more cumbersome to use. The rule is now to
only pass arguments that might help in recovering from the exception or
provide information that is otherwise not easily accessible. file: [fa59054e17] check-in: [3d16a30f41] user: js, branch: trunk, size: 14753 [annotate] [blame] [check-ins using] [diff]

The only places where BOOL is left are those where they are required by
the ABI. file: [4a71698195] check-in: [c5ef582958] user: js, branch: trunk, size: 10982 [annotate] [blame] [check-ins using] [diff]

It is now called -[numberOfBytesInReadBuffer].

Additionally, this commit renames OFStream's _cache to _readBuffer. file: [8b8198d6dd] check-in: [efe7be259d] user: js, branch: trunk, size: 10970 [annotate] [blame] [check-ins using] [diff]

This also implements chunked transfer encoding for OFHTTPServer and adds
a property for the protocol version to OFHTTPRequest(Reply).

Additionally, this makes it possible to add an async version of
OFHTTPClient. file: [9a386c8c54] check-in: [bdf9c4d96b] user: js, branch: trunk, size: 10827 [annotate] [blame] [check-ins using] [diff]

The class also has its own file now. file: [ba8461807d] check-in: [75f187cef5] user: js, branch: trunk, size: 12268 [annotate] [blame] [check-ins using] [diff]

This removes of_http_client_tls_socket_class in favor of having one
symbol for all places where TLS would be desired. file: [f1da42cb60] check-in: [1abe5a111f] user: js, branch: trunk, size: 12344 [annotate] [blame] [check-ins using] [diff]