ObjFW  History of src/OFString.m of fa0f45f5f8f5b9de

In order to not accidentally have C strings with \0, an
OFInvalidEncodingException is thrown when trying to get a C string for a
string that contains \0.

In order to get a C string with \0 anyway, a new method
-[insecureCStringWithEncoding:] is added. file: [235c6ba0c3] check-in: [6fbc3b18c7] user: js, branch: trunk, size: 70859 [annotate] [blame] [check-ins using] [diff]

It has been moved to the "unicode-normalization" branch until it is
ready. file: [70fe1f9064] check-in: [149874777c] user: js, branch: trunk, size: 64779 [annotate] [blame] [check-ins using] [diff]

While the idea sounds nice that the tag name is the class, this means the
serialization includes whether something is mutable or immutable. This means
doing as much as making something immutable changes the serialization, which
can then cause issues after being deserialized. file: [6c5b7de32e] check-in: [6ce0093f8d] user: js, branch: trunk, size: 65108 [annotate] [blame] [check-ins using] [diff]

The same can be achieved using -[OFString stringByAppendingString:] and
swapping the order of arguments. file: [9ff5525c3f] check-in: [114576fe94] user: js, branch: trunk, size: 66249 [annotate] [blame] [check-ins using] [diff]

Embedded files are available via the objfw-embedded: URL scheme. file: [0292b59c78] check-in: [1483709ab2] user: js, branch: trunk, size: 66556 [annotate] [blame] [check-ins using] [diff]

Even when there is neither files nor sockets, a scheme could still be
registered, for example for files embedded into the binary. file: [ff9b0b5a48] check-in: [1a49ef773d] user: js, branch: trunk, size: 66951 [annotate] [blame] [check-ins using] [diff]

This is only migrating some places, others will be migrated as they are
touched. file: [cc999ec09b] check-in: [a0fd103a0b] user: js, branch: trunk, size: 67153 [annotate] [blame] [check-ins using] [diff]

strtod() on HP-UX does not accept NaN. file: [3d251eedbb] check-in: [cd4fd689d6] user: js, branch: trunk, size: 67572 [annotate] [blame] [check-ins using] [diff]

It requires errno and every function that uses errno needs to be passed
from the linklib to the Amiga .library, so reducing usage of functions
that require errno is desirable. file: [68a16bb73e] check-in: [08ab815c2c] user: js, branch: trunk, size: 67075 [annotate] [blame] [check-ins using] [diff]

The new names should be more accurate. file: [8c96d0cf6e] check-in: [627511b032] user: js, branch: trunk, size: 65922 [annotate] [blame] [check-ins using] [diff]

In particular, this means that if initialization fails, the buffer is
NOT free'd. While freeing it on error in convenient when not handling
the error, it makes handling the error and retrying impossible (e.g.
retrying with a different encoding). file: [7044157852] check-in: [0f260e0fd5] user: js, branch: trunk, size: 65913 [annotate] [blame] [check-ins using] [diff]

While -[OFData initWithDataNoCopy:count:itemSize:freeWhenDone:] can
currently never throw, it might in the future, and when it does so in
the future, these would be memory leaks then. file: [ab9d9164f8] check-in: [a2b4238850] user: js, branch: trunk, size: 65979 [annotate] [blame] [check-ins using] [diff]

-[allocMemoryWithSize:] has book keeping overhead that is unnecessary
for temporary memory. file: [8da779313c] check-in: [0ddaac3f5b] user: js, branch: trunk, size: 65372 [annotate] [blame] [check-ins using] [diff]

While it makes sense to wrap malloc and calloc to replace the error
checking with exceptions, it does not make sense to wrap free. file: [4df757c93c] check-in: [3d8286feee] user: js, branch: trunk, size: 65456 [annotate] [blame] [check-ins using] [diff]

The internal hash is still 32 bit in most places, but this way, it is at
least not baked into the API and ABI and can be upgraded later, should
that ever be necessary. file: [91d4470735] check-in: [aeb403a1ed] user: js, branch: trunk, size: 65567 [annotate] [blame] [check-ins using] [diff]

amiga-gcc now has strto(u)ll. file: [f69742e62e] check-in: [5f494eaf81] user: js, branch: trunk, size: 65479 [annotate] [blame] [check-ins using] [diff]

It is not guaranteed that errno is set to 0 when there is no
overflow/underflow, so it needs to be set to 0 before, in case it
already is ERANGE from a previous call. file: [3a36a1ae89] check-in: [04bb18457a] user: js, branch: trunk, size: 65475 [annotate] [blame] [check-ins using] [diff]

Some implementations define this as a macro that indexes into a table,
where this would result in a negative index if char is signed. file: [fbf885bed7] check-in: [ce5dfd4a83] user: js, branch: trunk, size: 65427 [annotate] [blame] [check-ins using] [diff]

It is not guaranteed that a type encoding for it exists. file: [a680f97b80] check-in: [d74f244ab7] user: js, branch: trunk, size: 65352 [annotate] [blame] [check-ins using] [diff]

This solves the old signed vs. unsigned problem and allows for more
bases than just 8, 10 and 16, as well as auto-detection of the base (if
base is 0). file: [d974e2e6b6] check-in: [b6ee372b98] user: js, branch: trunk, size: 65344 [annotate] [blame] [check-ins using] [diff]

This requires a slight API change to OFWindowsRegistryKey, as the
previous API was depending on RegGetValueW(), which is not available on
Windows XP. file: [1a087004fb] check-in: [94f1f0f479] user: js, branch: trunk, size: 66760 [annotate] [blame] [check-ins using] [diff]

This fixes --disable-files. file: [77745cd7d6] check-in: [ad3fc136ba] user: js, branch: trunk, size: 68316 [annotate] [blame] [check-ins using] [diff]

The reason for this is that, in theory, this also supports versions
older than 3. file: [6b9d12d673] check-in: [5831626f68] user: js, branch: trunk, size: 67053 [annotate] [blame] [check-ins using] [diff]

Note: There is still some weirdness in the compiler. Instead of
m68k-amigaos-gcc, you need to use m68k-amigaos-g++ as OBJC, as otherwise
exceptions don't work.

Before running the tests, you need to run "stack 8192". file: [2d7f2091d4] check-in: [49cb6ada5f] user: js, branch: trunk, size: 67041 [annotate] [blame] [check-ins using] [diff]

This is the first step to have different versions of those for different
operating systems, rather than #ifdefs everywhere.

This also has the nice side-effect of not having one implementation in
OFString and another one in OFString_UTF8 anymore. The one in OFString
was a generic version, while the one in OFString_UTF8 was one optimized
for UTF-8 strings. Now only the version optimized for UTF-8 strings
exists, as this is by far the most common string implementation used,
and the overhead of converting from something else to UTF-8 is not more
than converting to UTF-32. file: [1c64791f79] check-in: [7a488c3062] user: js, branch: trunk, size: 65836 [annotate] [blame] [check-ins using] [diff]

When using self = [self init…], this already releases self on an
exception, so using it inside of a @try / @catch block that releases
self would lead to a double-release. file: [b00db1bc09] check-in: [81156ee75a] user: js, branch: trunk, size: 70307 [annotate] [blame] [check-ins using] [diff]

There is no reliable way to check if C++ already defined those, as Clang
now defines char{16,32}_t even when it is not in C++ 11 mode. So we
cannot check if we are not in C++ 11 and then define them and otherwise
rely on the C++ 11 types. file: [8f9360fade] check-in: [505137f25f] user: js, branch: trunk, size: 69675 [annotate] [blame] [check-ins using] [diff]

Since OFComparing no longer conforms to OFObject, OFObject needed to be
added at a few places. file: [27fbf1ebf8] check-in: [11f24addd7] user: js, branch: trunk, size: 69184 [annotate] [blame] [check-ins using] [diff]

This now prevents double slashes / backslashes. file: [90e49f1e77] check-in: [80ac05eb8f] user: js, branch: trunk, size: 68846 [annotate] [blame] [check-ins using] [diff]

This removes all query methods and replaces them by a single one that
returns a dictionary. Not only does this allow querying multiple
attributes at once, it is also more extensible, as the previous
attributes were specific to file URLs. file: [02ef45c9df] check-in: [34fe205b84] user: js, branch: trunk, size: 67076 [annotate] [blame] [check-ins using] [diff]

This also adds -[OFURL fileSystemRepresentation] to correctly convert a
URL back to a path in the local file system. file: [e0c9985c5f] check-in: [b83d1414b1] user: js, branch: trunk, size: 66069 [annotate] [blame] [check-ins using] [diff]

Instead, explicitly declare them, as OF_ASSUME_NONNULL_{BEGIN,END} does
not apply to implicit return types. This means that after this commit,
all init methods have a nonnull return type, as they should have. file: [0cbd68fca6] check-in: [2f4e0df8be] user: js, branch: trunk, size: 66347 [annotate] [blame] [check-ins using] [diff]

This does not make OFHTTPClientResponse completely asynchronous yet.

Since -[initWithURL:] does not work well with an asynchronous API, it no
longer supports HTTP URLs. However, a new, asynchronous API will be
added as a replacement later on. file: [661e4352a9] check-in: [56a18442e2] user: js, branch: trunk, size: 65583 [annotate] [blame] [check-ins using] [diff]

This removes "b" for binary and always uses binary, as there is no good
reason to not use binary. file: [18c9b8230b] check-in: [23e57c5040] user: js, branch: trunk, size: 65349 [annotate] [blame] [check-ins using] [diff]

This reduces the ifdef hell and makes much more sense, as with ixemul,
no native calls are allowed. file: [51b9feaad9] check-in: [c9621825fc] user: js, branch: trunk, size: 65474 [annotate] [blame] [check-ins using] [diff]

Everything else should just use OFFileManager. file: [094992d7f8] check-in: [4ebeb7e3a2] user: js, branch: trunk, size: 64620 [annotate] [blame] [check-ins using] [diff]

This matches Apple's style. file: [37a16908bb] check-in: [6b77a5dd8b] user: js, branch: trunk, size: 64571 [annotate] [blame] [check-ins using] [diff]

Casts are now written like types in variable declarations. file: [e0471f5f58] check-in: [4af49a13c3] user: js, branch: trunk, size: 64664 [annotate] [blame] [check-ins using] [diff]

This allows building a smaller ObjFW if those are not needed. file: [25043e1af3] check-in: [766eedef4c] user: js, branch: trunk, size: 62775 [annotate] [blame] [check-ins using] [diff]

This only differs from codepage 850 by a single code point. file: [3404c5b7b6] check-in: [adbf31f2f9] user: js, branch: trunk, size: 60140 [annotate] [blame] [check-ins using] [diff]

Forgot to add 2017, even though I already did quite some changes in
2017. file: [0f58e25244] check-in: [44f45c2e35] user: js, branch: trunk, size: 59318 [annotate] [blame] [check-ins using] [diff]

This singleton gives access to all things locale, including the ability
to get localized strings.

This also adds the OF_LOCALIZED() macro. Its first argument is an ID for
the string to be localized and its second argument is the fallback
string to be used if it cannot retrieve the localized string. Following
that are variable name / value pairs to be replaced in the localized
string.

Getting translated strings is not implemented yet: Instead, it always
uses the fallback string.

This also switches ofhttp to localized strings. file: [451da8b217] check-in: [06bcb21fc7] user: js, branch: trunk, size: 59312 [annotate] [blame] [check-ins using] [diff]

This avoids the hacks introduced in the previous commit, if those
functions are available. file: [011cdaca0e] check-in: [0ad678f125] user: js, branch: trunk, size: 59306 [annotate] [blame] [check-ins using] [diff]

This is achieved by replacing the locale's decimal point with "." after
formatting and replacing "." with the locale's decimal point before
parsing.

To still use the decimal point from the locale for formatting, the new
flag "," is introduced to formats. This is useful for just printing a
string to the user that is not saved to a file or sent via a network.

While this is an ugly hack, there is no better way to do this other than
implementing the functionality of printf and strtod myself, as POSIX
does not specify versions of these that take a locale as an argument.
While this is a lot of work and error-prone, I will most likely end up
doing this eventually.

This commit also enables the locale in OFApplication to notice when
things break. As a nice side effect, ofhttp now uses the locale's
decimal point in its user interface. file: [2a34be912f] check-in: [6dff0f5922] user: js, branch: trunk, size: 58336 [annotate] [blame] [check-ins using] [diff]

These are independent of the locale and work on the ASCII character set.

Unlike the C ones, these are 8-bit safe, meaning if a character > 0x7F
is passed, is{alpha,alnum} returns false and to{upper,lower} returns the
original character. file: [4bcce6446e] check-in: [d9eb7b50b3] user: js, branch: trunk, size: 58031 [annotate] [blame] [check-ins using] [diff]

This is only useful for size-constrained systems and changes the
behavior as follows:

* Case-insensitive comparisons only work as expected for ASCII
characters. This means that while a and A compare equal, ä and Ä do
not.

* -[OFMutableString {upper,lower}] and
-[OFString {upper,lower}caseString] only work on ASCII characters;
all other characters are left as they are. This means that the upper
version of a is A, but the upper version of ä is still ä (and vice
versa for lower). file: [719f50f06e] check-in: [b7b33247d6] user: js, branch: trunk, size: 58106 [annotate] [blame] [check-ins using] [diff]

OFHash is too generic, as this protocol is for cryptographic hashes. It
also solves the ambiguity of the +[hash] method (which is now
+[cryptoHash]), which conflicted with -[hash] on the class itself,
resulting in classes conforming to OFHash not being able to be used as a
key in an OFDictionary. file: [4050f6aba6] check-in: [951f530aee] user: js, branch: trunk, size: 57989 [annotate] [blame] [check-ins using] [diff]

This is required for static linking, as otherwise
+[OFStdIOStream_Win32Console load] will not be called. file: [63687acf9a] check-in: [e53a4fd61c] user: js, branch: trunk, size: 57684 [annotate] [blame] [check-ins using] [diff]

It now returns <= 0 on error, with negative values being the number of
bytes it would have needed * -1.

This can be used to detect cut off and how many bytes are missing. file: [b713fffc4b] check-in: [5e2ef97c35] user: js, branch: trunk, size: 56129 [annotate] [blame] [check-ins using] [diff]

While at it, also update the mail address. file: [b888341646] check-in: [cec0f072f8] user: js, branch: 0.8, size: 56231 [annotate] [blame] [check-ins using] [diff]

While at it, also update the mail address. file: [63a41717d2] check-in: [2a27cf3000] user: js, branch: trunk, size: 56018 [annotate] [blame] [check-ins using] [diff]

Now that we can require GCC >= 4.6, we no longer need to fall back to
using a category on old compilers. file: [f76a1b5752] check-in: [2aca549d60] user: js, branch: trunk, size: 56014 [annotate] [blame] [check-ins using] [diff]

Now that we require GCC >= 4.6 anyway, there's no more reason to not use
it anymore. file: [01f9bb127a] check-in: [6b13727ce0] user: js, branch: trunk, size: 56033 [annotate] [blame] [check-ins using] [diff]

The old behaviour where the exception would access errno directly on
creation of the exception was very fragile. The two main problems with
it were that sometimes it would pick up an errno even though none had
been set and in other cases that when the exception was created errno
had already been overridden.

This also greatly increases errno handling on Win32, especially in
conjunction with sockets. It can still be improved further, though. file: [52d4d85f15] check-in: [62e2de30b9] user: js, branch: trunk, size: 55971 [annotate] [blame] [check-ins using] [diff]

This replaces +[OFString nativeOSEncoding]. file: [f631bc6b9b] check-in: [ec66e49dca] user: js, branch: trunk, size: 55944 [annotate] [blame] [check-ins using] [diff]

This means that OFObject.h imports macros.h now, making it unnecessary
to manually import macros.h in almost every file. And while at it, also
import autorelease.h in OFObject.h, so that this doesn't need to be
manually imported in almost every file as well. file: [c9e153f6dd] check-in: [13ee56edf3] user: js, branch: trunk, size: 56115 [annotate] [blame] [check-ins using] [diff]

No functional change intended. file: [3cae95280a] check-in: [9da9b2b64c] user: js, branch: trunk, size: 56030 [annotate] [blame] [check-ins using] [diff]

This also adds OF_JSON_REPRESENTATION_PRETTY. file: [006b401a4b] check-in: [eaa612ad50] user: js, branch: trunk, size: 55435 [annotate] [blame] [check-ins using] [diff]

This replaces OF_STRING_ENCODING_NATIVE, as a define was not flexible
enough (determining the native OS encoding at startup was not possible). file: [548eca2146] check-in: [e54c8c0368] user: js, branch: trunk, size: 55470 [annotate] [blame] [check-ins using] [diff]

The port was much easier than expected. The only changes that were
required were defining the missing lrint() and useconds_t, the rest was
optional (like using backslashes for paths, even though slashes work
fine). file: [95256ec388] check-in: [a0b63a8df4] user: js, branch: trunk, size: 55950 [annotate] [blame] [check-ins using] [diff]

This removes -[initWithPath:,...], so that it's finally possible to use
-[initWithPath:] for many places where a workaround like
-[initWithDirectoryPath:] was required before. file: [6989417794] check-in: [7c4179ab9b] user: js, branch: trunk, size: 55019 [annotate] [blame] [check-ins using] [diff]

This mostly removes the argument for the class in which the exception
occurred. As backtraces were recently added for all platforms, the
passed class does not give any extra information on where the exception
occurred anymore.

This also removes a few other arguments which were not too helpful. In
the past, the idea was to pass as many arguments as possible so that it
is easier to find the origin of the exception. However, as backtraces
are a much better way to find the origin, those are not useful anymore
and just make the exception more cumbersome to use. The rule is now to
only pass arguments that might help in recovering from the exception or
provide information that is otherwise not easily accessible. file: [dc3e1b10e9] check-in: [3d16a30f41] user: js, branch: trunk, size: 55435 [annotate] [blame] [check-ins using] [diff]

No support for extensions yet.

This obsoletes BinaryPack. file: [b10086fe78] check-in: [cba771824e] user: js, branch: trunk, size: 56601 [annotate] [blame] [check-ins using] [diff]

It still needs some more special casing for Windows. file: [efdad56a90] check-in: [379d75e1bc] user: js, branch: trunk, size: 56600 [annotate] [blame] [check-ins using] [diff]

The only places where BOOL is left are those where they are required by
the ABI. file: [76e87b40ed] check-in: [c5ef582958] user: js, branch: trunk, size: 55902 [annotate] [blame] [check-ins using] [diff]

This also implements chunked transfer encoding for OFHTTPServer and adds
a property for the protocol version to OFHTTPRequest(Reply).

Additionally, this makes it possible to add an async version of
OFHTTPClient. file: [537b9ea7bb] check-in: [bdf9c4d96b] user: js, branch: trunk, size: 54660 [annotate] [blame] [check-ins using] [diff]

For ObjC++, these are defined to char{16,32}_t if C++11 is used. This
means passing e.g. a C++11 UTF-16 string to -[initWithUTF16String:]
works without a warning.

For C11, this is not necessary, as char{16,32}_t are just typedef'd to
uint_least{16,32}_t in <uchar.h> and thus compatible to the definition
of of_char{16,32}_t. The reason for not defining of_char{16,32}_t to
char{16,32}_t when using C11 is that <uchar.h> might be missing and that
char{16,32}_t are not required to be UTF-{16,32}, whereas in C++, they
are required to be UTF-{16,32}. file: [3b195e39ec] check-in: [545b6f740e] user: js, branch: trunk, size: 50107 [annotate] [blame] [check-ins using] [diff]

This reverts 5362941.

Even though *UsingEncoding: is what Foundation uses, *WithEncoding: is
more natural and is more consistent with the rest. Also, this means this
restores it to the API of 0.7. file: [cb1b20524f] check-in: [9ddd33a258] user: js, branch: trunk, size: 50018 [annotate] [blame] [check-ins using] [diff]

With this change, there is a clear separation between characters and
UTF-32 strings now. Characters are just an array of characters in the
native endianess, with no BOM prepended. UTF-32 on the other hand may
have a BOM and can be swapped and is optionally zero-terminated.

This also fixes a few missing UTF-16 init methods in OFMutableString. file: [5a8a220443] check-in: [50916b8dbe] user: js, branch: trunk, size: 50023 [annotate] [blame] [check-ins using] [diff]

This partly reverts e8502c7.

The rationale behind this is that, on some OSes, native APIs (on e.g.
Windows) take UTF-16 strings that are zero-terminated.

However, forcing zero-termination for every string so that -[characters]
returns a zero-terminated UTF-32 string does not make sense. Therefore,
in the future, -[UTF32String] will be added, which will include
zero-termination. OFString subclasses can then just return their
internal representation if it includes a terminating zero or create a
copy that has a terminating zero if not. file: [6044846057] check-in: [9d3cd5e5fe] user: js, branch: trunk, size: 47681 [annotate] [blame] [check-ins using] [diff]

* Instead of Unicode "strings", it uses the term "Unicode character
array".
* No terminating 0 character anymore, as these are not C strings
anyway. file: [2366afaf49] check-in: [640b225ba7] user: js, branch: trunk, size: 45873 [annotate] [blame] [check-ins using] [diff]

The class also has its own file now. file: [f223cdd779] check-in: [75f187cef5] user: js, branch: trunk, size: 46591 [annotate] [blame] [check-ins using] [diff]

This adds ISO 8859-1 and ASCII, ISO 8859-15 and Windows-1252 will
follow. file: [29ef9c669e] check-in: [556234e290] user: js, branch: trunk, size: 45390 [annotate] [blame] [check-ins using] [diff]

This should make it more future-proof, as adding new options is easier
than adding new methods for each and every combination of options. file: [5e5459873a] check-in: [fcaa549324] user: js, branch: trunk, size: 43946 [annotate] [blame] [check-ins using] [diff]

This makes it clear that multiple evaluation of parameters is possible. file: [3d2b1f904a] check-in: [1cb8fee5c3] user: js, branch: trunk, size: 43092 [annotate] [blame] [check-ins using] [diff]

This greatly improves performance, as it gets rid of the overhead of
OFAutoreleasePool. file: [bc6ced3df8] check-in: [1255f3a11a] user: js, branch: trunk, size: 46045 [annotate] [blame] [check-ins using] [diff]

This is necessary for static linking. Without this, the runtime keeps
static instances of class OFConstantString forever in its list of
objects to initialize. This resulted in string literals still having
their isa set to Nil and thus a crash on the first dispatch. file: [e165bf4e50] check-in: [e810b8ab15] user: js, branch: runtime, size: 45339 [annotate] [blame] [check-ins using] [diff]

Also fix a bug where OFBigDataArray would waste memory. file: [788fd6f59d] check-in: [f7576a66ce] user: js, branch: trunk, size: 45289 [annotate] [blame] [check-ins using] [diff]

It is a class cluster now, but there is still much to improve. file: [c57b8a6b12] check-in: [301ee63212] user: js, branch: trunk, size: 41020 [annotate] [blame] [check-ins using] [diff]

This was changed from native to big endian because UCS-2 is defined to
be big endian. file: [1d69719094] check-in: [f412995e6a] user: js, branch: trunk, size: 43212 [annotate] [blame] [check-ins using] [diff]

It turns out the overhead of calling madvise() is bigger than the gain
of using it. file: [f8793e2293] check-in: [6cbbd7c39c] user: js, branch: trunk, size: 43123 [annotate] [blame] [check-ins using] [diff]

This is safe as an "exception loop" can't happen, since if allocating
an exception fails, it throws an OFAllocFailedException which is
preallocated and can always be thrown.

So, the worst case would be that an autorelease of an exception fails,
triggering an OFOutOfMemoryException for which there is no memory,
resulting in an OFAllocFailedException to be thrown. file: [93030ae4fe] check-in: [e1e7ffa903] user: js, branch: trunk, size: 43700 [annotate] [blame] [check-ins using] [diff]

Also removes -[appendCStringWithoutUTF8Checking:] as it is dangerous and
not compatible with this optimization. file: [fee37530e3] check-in: [36e8a94f34] user: js, branch: trunk, size: 43190 [annotate] [blame] [check-ins using] [diff]

This way, we don't need a new parser and don't use a new format. file: [47a649e523] check-in: [494dca4ae4] user: js, branch: trunk, size: 34751 [annotate] [blame] [check-ins using] [diff]