logo Blog feeds

Nim BlogVersion 0.20.2 released

The Nim team is happy to announce version 0.20.2, which is our second release candidate for version 1.0.

To read more about version 0.20 and our plans for version 1.0 and beyond, check the previous release notes. This is mostly a bugfix release of version 0.20.0 (1.0 RC1).

The only new feature is that toOpenArray, our slicing operator, is now available for the JavaScript backend.

Although this release comes only one month after a previous release, it has over 200 new commits, fixing over 70 reported issues and bringing lots of improvements which should make Nim even more stable.

Style checks

Since version 0.20.0 we improved our style checker to the point that Nim now bootstraps with --styleCheck:error flag, meaning that the compiler and the standard library are now style checked and they won’t compile if different styles (e.g. fooBar and foo_bar) are used.

If you would like to only style check your own nimble package, but not its dependencies, you can use --styleCheck:hint, making your package consistent while still retaining perfect interoperability with all other Nim code in the wild.

Nimpretty

One of the areas which we focused on for this release is our nimpretty tool – it is a Nim source code beautifier, to format code according to the official style guide (NEP-1). More than 30 commits were made aiming to improve nimpretty experience and to make it more robust for real-world scenarios. Nimpretty version 0.2 ships with Nim, and we invite you to give it a try.

New runtime

The new runtime is progressing nicely but it is off the critical path for version 1.0 as it’s an opt-in feature. The next significant milestone is to make the new runtime work with async. Only after this is accomplished we can push for more adoption of it.

Installing 0.20.2

If you have installed a previous version of Nim using choosenim, getting Nim 0.20.2 is as easy as:

$ choosenim update stable

If you don’t have it already, you can get choosenim by following these instructions or you can install Nim by following the instructions on our install page.

Changes affecting backwards compatibility

  • All strutils.rfind procs now take start and last like strutils.find with the same data slice/index meaning. This is backwards compatible for calls not changing the rfind start parameter from its default. (#11487)

    In the unlikely case that you were using rfind X, start=N, or rfind X, N, then you need to change that to rfind X, last=N or rfind X, 0, N. (This should minimize gotchas porting code from other languages like Python or C++.)

  • On Windows stderr/stdout/stdin are not opened as binary files anymore. Use the switch -d:nimBinaryStdFiles for a transition period.

Breaking changes in the standard library

  • Mac OS X / BSD: TSa_Family is now the uint8 type, so type conversions like x.sin_family = uint16 toInt(nativesockets.AF_INET) need to be changed into x.sin_family = TSa_Family toInt(nativesockets.AF_INET).

Library additions

  • toOpenArray is now available for the JS target.

Library changes

  • Fix async IO operations stalling even after socket is closed. (#11232)

  • More informative error message for streams.openFileStream. (#11438)

Compiler changes

  • Better error message for IndexError for empty containers. (#11476)

  • Fix regression in semfold for old right shift. (#11477)

  • Fix for passing tuples as static params to macros. (#11423)

Bugfixes

  • Fixed “nimpretty goes crazy with this snippet” (#10295)
  • Fixed “nimpretty doesn’t trim all whitespace on the right side of an export marker” (#10177)
  • Fixed “nimpretty gives invalid indentation to array elements” (#9505)
  • Fixed “nimpretty doesn’t indent correctly if preceding line ends with ;, { etc” (#10159)
  • Fixed “Nimpretty wrong indentation for doc comments” (#10156)
  • Fixed “HttpClient Documentation needs Proxy example” (#11281)
  • Fixed “nimpretty aligns comment annoyingly” (#9399)
  • Fixed “ENDB missing from Docs and TheIndex, remove ENDB from –fullhelp” (#11431)
  • Fixed “Nimrod on Documentation generator” (#11460)
  • Fixed “nimpretty (de)indents code where it shouldn’t” (#11468)
  • Fixed “nimpretty adds spaces where it shouldn’t” (#11470)
  • Fixed “nimpretty formats enums differently based on commas” (#11467)
  • Fixed “strutils.rfind start parameter is unecessarily unusual” (#11430)
  • Fixed “TinyC is not documented” (#11495)
  • Fixed “c2nim missing from 0.20.0” (#11434)
  • Fixed “nimsuggest doesn’t work unless compiled with -d:danger” (#11482)
  • Fixed “random.initRand crashes in Nim 0.20.0 JS backend” (#11450)
  • Fixed “0.20.0 fails to bootstrap when passing “–verbosity:2” (or 3) to koch” (#11436)
  • Fixed “Destructors lifting doesn’t work with inheritance” (#11517)
  • Fixed “std/sums missing from TheIndex” (#11543)
  • Fixed “sequtils module: link is broken” (#11546)
  • Fixed “Case Statement Macros do not work in functions above exported functions.” (#11556)
  • Fixed “newruntime: internal error when initializing a proc variable” (#11533)
  • Fixed “newruntime: error when modifying a sequence” (#11524)
  • Fixed “fmod and other math module functions are missing in js mode” (#4630)
  • Fixed “Object variants and new runtime” (#11563)
  • Fixed “newruntime exceptions” (#11577)
  • Fixed “nimpretty is not aware that the next line is a part of the same context” (#11469)
  • Fixed “Distinct procs fail to compile” (#11600)
  • Fixed “[SharedTables] Error: undeclared identifier: ‘defaultInitialSize’” (#11588)
  • Fixed “newSeqOfCap is not working in newruntime” (#11098)
  • Fixed “nimpretty destroys source with a source code filter” (#11532)
  • Fixed “Unexpected behaviour when constructing with result” (#11525)
  • Fixed “Regression in 0.20.0: Nested proc using outer scope variable fails to compile” (#11523)
  • Fixed “os:standalone Error: redefinition of ‘nimToCStringConv’ “ (#11445)
  • Fixed “No ambiguity error on field overloaded by field=” (#11514)
  • Fixed “object variants and new runtime part 2” (#11611)
  • Fixed “seq Error: unhandled exception: value out of range: 32772 “ (#11606)
  • Fixed “Compiled binary includes full path to internal nim files” (#11572)
  • Fixed “Newruntime: top-level string variable is empty after an array assignment” (#11614)
  • Fixed “Newruntime: raise ObjContructor() doesn’t compile” (#11628)
  • Fixed “Owned ref can be copied and causes double-free” (#11617)
  • Fixed “When compiling to JS (in -d:release) output contains toolchain path on dev’s machine” (#11545)
  • Fixed “wrong unicode string output” (#11618)
  • Fixed “unittest “generic instantiation too nested” error” (#11515)
  • Fixed “Last read of a var parameter generates sink instead of assignment” (#11633)
  • Fixed “const table with proc type does not compile anymore” (#11479)
  • Fixed “Can’t use offsetOf on object that ends with an UncheckedArray” (#11320)
  • Fixed “Newruntime/regression: Bad codegen for inline methods” (#11636)
  • Fixed “String or sequences inside a loop are not cleared in each iteration” (#11510)
  • Fixed “Nim -v doesn’t acknowledge new danger flag” (#11484)
  • Fixed “SIGSEGV while compiling when trying to instantiate a case type, that uses an enum with non-zero start” (#11585)
  • Fixed “Compile-time usage of parts of strutils fails when using -d:useNimRtl” (#8405)
  • Fixed “regression(0.20): finally block executed twice in VM” (#11610)
  • Fixed “exportc symbol not exported, leading to link error” (#11651)
  • Fixed “Render bug: opearator priority can be wrong for second argument” (#11662)
  • Fixed “Nim & C disagree on type size” (#6860)
  • Fixed “Spawn passes refs across threads” (#7057)
  • Fixed “BUG: “varargs[string, $]” calls $ n^2 times instead of n times (n=len(varargs))” (#8316)
  • Fixed “Problem with the same name for module and exported type” (#3333)
  • Fixed “–gc:go does not work anymore” (#11447)
  • Fixed “Error: inconsistent typing for reintroduced symbol” (#11494)
  • Fixed “Auto generated destructors for tuples/objects fail with “non-trivial” error…” (#11671)
  • Fixed “Regression: parameter default value + typedesc parameter causes compiler crash” (#11660)
  • Fixed “newruntime: undetected dangling ref” (#11350)
  • Fixed “Newruntime: setLen() not working on sequences of owned refs” (#11530)
  • Fixed “Incorrect overflow/underflow error in case statements” (#11551)
  • Fixed “cgen preprocessor directive placed after struct declaration” (#11691)
  • Fixed “continue in an except Exception as e block crashes the compiler.” (#11683)
  • Fixed “nimsuggest, nim check segfault when using bindSym and doAssert” (#10901)
  • Fixed “Nimpretty issue, multiline string of x length only causes output grow every time.” (#11700)
  • Fixed “Array indexed by distinct int doesnt work with iterators” (#11715)

Nim BlogVersion 0.20.0 released

We are very proud to announce Nim version 0.20.

This is a massive release, both literally and figuratively. It contains more than 1,000 commits and it marks our release candidate for version 1.0!

Version 0.20 introduces a number of breaking changes required for version 1.0. These are changes that we feel have been necessary to include in Nim 1.0 and we currently have no plans for any further breaking changes. Version 0.20 is effectively Nim 1.0 RC1.

Why not just release v1.0? We want to give the community a chance to test 0.20.0 and find bugs that may require breaking changes. This is unlikely, but may require further release candidates. Make no mistake, this release represents what we consider Nim 1.0, we have been working towards this milestone for many years and are incredibly excited to reach it.

The stability guarantee

A 1.0 release means that once either Nim 0.20.0 is promoted to 1.0 status, or another release candidate is, there will no longer be any breaking changes made. Version 1.0 will become a long-term supported stable release that will only receive bug fixes and new features in the future, as long as they don’t break backwards compatibility.

The 1.0.x branch will receive bug fixes for as long as there is demand for them. New features (which do not break backwards compatibility) will continue in steadily advancing 1.x branches.

Our goal is to make sure that code which compiled under Nim 1.0 continues to compile under any stable Nim 1.x version.

What’s included under the stability guarantee?

Backwards compatibility covers only the stable fragment of the language, as defined by the manual.

The compiler still implements experimental features which are documented in the newly written “experimental manual”, these features are subject to changes which may be backwards incompatible, some of the features included under this umbrella are concepts, the do notation and a few others. Be wary of using these features in production, but do get in touch with us if you want to learn more about our plans regarding them.

The standard library is also covered, we will continue to deprecate procedures if we must, but they will remain supported throughout the 1.x version series.

Exceptions to the rule

We of course have to concede that there are exceptions. In certain serious cases, like for example when a security vulnerability is discovered in the stdlib, we reserve the right to break code which uses it.

Installing 0.20.0

If you have installed a previous version of Nim using choosenim, getting Nim 0.20.0 is as easy as:

$ choosenim update stable

If you don’t have it already, you can get choosenim by following these instructions or you can install Nim by following the instructions on our install page.

Nimble 0.10.2

This release includes a brand new version of Nimble. See Nimble changelog.

Contributors to v0.20

Our contributors are amazing, and there is far too many to list here. Big thanks to all of you, we couldn’t have pulled off this release without you!

New features

Version 0.20 is positively jam packed with features. Here is a selection of our favourites:

not is always a unary operator

let a = false

# v0.19:
assert not a # Error: type mismatch: got <proc (cond: untyped, msg: string): typed, bool>
assert(not a) # workaround

# v0.20:
assert not a

Stricter compile time checks for integer and float conversions

# v0.19:
const b = uint16(-1)
echo b # 65535

# v0.20:
const b = uint16(-1)
# Error: -1 can't be converted to uint16
const c = not uint16(0)
echo c # 65535

Tuple unpacking for constant and for loop variables

const (d, e) = (7, "eight")
# v0.19: Error: identifier expected, but got '('

# v0.20:
echo d # 7
echo e # eight


let f = @[(51, 10), (23, 25)]

for (x, y) in f: # v0.19: Error: identifier expected, but got '('
  echo x + y
# v0.20:
# 61
# 48

Hash sets and tables are initialized by default

import sets, tables

var s: HashSet[int]

s.incl(5)
# v0.19: `isValid(s)` Error: unhandled exception: The set needs to be initialized. [AssertionError]
# v0.20:
echo s # {5}


var t: Table[char, int]
t['a'] = 10
# v0.19: Error: unhandled exception: index out of bounds [IndexError]
# v0.20:
echo t # {'a': 10}

Better error message for case-statements

type
  MyEnum = enum
    first
    second
    third
    fourth

proc foo(x: MyEnum): int =
  case x
  of first: 1
  of second: 2
  of third: 3
  of fourth: 4
  else: 99

# v0.19: compiles
# v0.20: Error: invalid else, all cases are already covered


proc bar(x: MyEnum): int =
  case x
  of first: 1
  of third: 3

# v0.19: Error: not all cases are covered
# v0.20: Error: not all cases are covered; missing: {second, fourth}

The length of a table must not change during iteration

import tables

var xs = {1: "one", 2: "two", 3: "three"}.toTable

for x in xs.keys:
  if x mod 2 == 0:
    xs[10*x] = "a lot"
echo xs

# v0.19: {200: "a lot", 1: "one", 2: "two", 3: "three", 20: "a lot"}
# v0.20: Error: unhandled exception: the length of the table changed while iterating over it [AssertionError]

Better error message for index out of bounds

let a = [10, 20, 30]

echo a[5]
# v0.19: Error: index out of bounds
# v0.20: Error: index 5 not in 0 .. 2

Changelog

Changes affecting backwards compatibility

  • shr is now sign preserving. Use -d:nimOldShiftRight to enable the old behavior globally.

  • The isLower, isUpper family of procs in strutils/unicode operating on strings have been deprecated since it was unclear what these do. Note that the much more useful procs that operate on char or Rune are not affected.

  • strutils.editDistance has been deprecated, use editdistance.editDistance or editdistance.editDistanceAscii instead.

  • The OpenMP parallel iterator `||` now supports any #pragma omp directive and not just #pragma omp parallel for. See OpenMP documentation.

    The default annotation is parallel for, if you used OpenMP without annotation the change is transparent, if you used annotations you will have to prefix your previous annotations with parallel for.

    Furthermore, an overload with positive stepping is available.

  • The unchecked pragma was removed, instead use system.UncheckedArray.

  • The undocumented #? strongSpaces parsing mode has been removed.

  • The not operator is now always a unary operator, this means that code like assert not isFalse(3) compiles.

  • getImpl on a var or let symbol will now return the full IdentDefs tree from the symbol declaration instead of just the initializer portion.

  • Methods are now ordinary “single” methods, only the first parameter is used to select the variant at runtime. For backwards compatibility use the new --multimethods:on switch.

  • Generic methods are now deprecated; they never worked well.

  • Compile time checks for integer and float conversions are now stricter. For example, const x = uint32(-1) now gives a compile time error instead of being equivalent to const x = 0xFFFFFFFF'u32.

  • Using typed as the result type in templates/macros now means “expression with a type”. The old meaning of typed is preserved as void or no result type at all.

  • A bug allowed macro foo(): int = 123 to compile even though a macro has to return a NimNode. This has been fixed.

  • With the exception of uint and uint64, conversion to unsigned types are now range checked during runtime.

  • Macro arguments of type typedesc are now passed to the macro as NimNode like every other type except static. Use typed for a behavior that is identical in new and old Nim. See the RFC Pass typedesc as NimNode to macros for more details.

Breaking changes in the standard library

  • osproc.execProcess now also takes a workingDir parameter.

  • std/sha1.secureHash now accepts openArray[char], not string. (Former successful matches should keep working, though former failures will not.)

  • options.UnpackError is no longer a ref type and inherits from system.Defect instead of system.ValueError.

  • system.ValueError now inherits from system.CatchableError instead of system.Defect.

  • The procs parseutils.parseBiggestInt, parseutils.parseInt, parseutils.parseBiggestUInt and parseutils.parseUInt now raise a ValueError when the parsed integer is outside of the valid range. Previously they sometimes raised an OverflowError and sometimes they returned 0.

  • The procs parseutils.parseBin, parseutils.parseOct and parseutils.parseHex were not clearing their var parameter number and used to push its value to the left when storing the parsed string into it. Now they always set the value of the parameter to 0 before storing the result of the parsing, unless the string to parse is not valid (then the value of number is not changed).

  • streams.StreamObject now restricts its fields to only raise system.Defect, system.IOError and system.OSError. This change only affects custom stream implementations.

  • nre’s RegexMatch.{captureBounds,captures}[] no longer return Option or nil/"", respectively. Use the newly added n in p.captures method to check if a group is captured, otherwise you’ll receive an exception.

  • nre’s RegexMatch.{captureBounds,captures}.toTable no longer accept a default parameter. Instead uncaptured entries are left empty. Use Table.getOrDefault() if you need defaults.

  • nre’s RegexMatch.captures.{items,toSeq} now returns an Option[string] instead of a string. With the removal of nil strings, this is the only way to indicate a missing match. Inside your loops, instead of capture == "" or capture == nil, use capture.isSome to check if a capture is present, and capture.get to get its value.

  • nre’s replace() no longer throws ValueError when the replacement string has missing captures. It instead throws KeyError for named captures, and IndexError for unnamed captures. This is consistent with RegexMatch.{captureBounds,captures}[].

  • splitFile now correctly handles edge cases, see #10047.

  • isNil is no longer false for undefined in the JavaScript backend: now it’s true for both nil and undefined. Use isNull or isUndefined if you need exact equality: isNil is consistent with ===, isNull and isUndefined with ==.

  • several deprecated modules were removed: ssl, matchers, httpserver, unsigned, actors, parseurl

  • two poorly documented and not used modules (subexes, scgi) were moved to graveyard (they are available as Nimble packages)

  • procs string.add(int) and string.add(float) which implicitly convert ints and floats to string have been deprecated. Use string.addInt(int) and string.addFloat(float) instead.

  • case object branch transitions via system.reset are deprecated. Compile your code with -d:nimOldCaseObjects for a transition period.

  • base64 module: The default parameter newLine for the encode procs was changed from "\13\10" to the empty string "".

Breaking changes in the compiler

  • The compiler now implements the “generic symbol prepass” for when statements in generics, see bug #8603. This means that code like this does not compile anymore:
proc enumToString*(enums: openArray[enum]): string =
  # typo: 'e' instead 'enums'
  when e.low.ord >= 0 and e.high.ord < 256:
    result = newString(enums.len)
  else:
    result = newString(enums.len * 2)
  • discard x is now illegal when x is a function symbol.

  • Implicit imports via --import: module in a config file are now restricted to the main package.

Library additions

  • There is a new stdlib module std/editdistance as a replacement for the deprecated strutils.editDistance.

  • There is a new stdlib module std/wordwrap as a replacement for the deprecated strutils.wordwrap.

  • Added split, splitWhitespace, size, alignLeft, align, strip, repeat procs and iterators to unicode.nim.

  • Added or for NimNode in macros.

  • Added system.typeof for more control over how type expressions can be deduced.

  • Added macros.isInstantiationOf for checking if the proc symbol is instantiation of generic proc symbol.

  • Added the parameter isSorted for the sequtils.deduplicate proc.

  • Added os.relativePath.

  • Added parseopt.remainingArgs.

  • Added os.getCurrentCompilerExe (implemented as getAppFilename at CT), can be used to retrieve the currently executing compiler.

  • Added xmltree.toXmlAttributes.

  • Added std/sums module for fast summation functions.

  • Added Rusage, getrusage, wait4 to the posix interface.

  • Added the posix_utils module.

  • Added system.default.

  • Added sequtils.items for closure iterators, allows closure iterators to be used by the mapIt, filterIt, allIt, anyIt, etc.

Library changes

  • The string output of macros.lispRepr proc has been tweaked slightly. The dumpLisp macro in this module now outputs an indented proper Lisp, devoid of commas.

  • Added macros.signatureHash that returns a stable identifier derived from the signature of a symbol.

  • In strutils empty strings now no longer match as substrings.

  • The Complex type is now a generic object and not a tuple anymore.

  • The ospaths module is now deprecated, use os instead. Note that os is available in a NimScript environment but unsupported operations produce a compile-time error.

  • The parseopt module now supports a new flag allowWhitespaceAfterColon (default value: true) that can be set to false for better Posix interoperability. (Bug #9619.)

  • os.joinPath and os.normalizePath handle edge cases like "a/b/../../.." differently.

  • securehash was moved to lib/deprecated.

  • The switch -d:useWinAnsi is not supported anymore.

  • In times module, procs format and parse accept a new optional DateTimeLocale argument for formatting/parsing dates in other languages.

Language additions

  • Vm support for float32<->int32 and float64<->int64 casts was added.
  • There is a new pragma block noSideEffect that works like the gcsafe pragma block.
  • added os.getCurrentProcessId.
  • User defined pragmas are now allowed in the pragma blocks.
  • Pragma blocks are no longer eliminated from the typed AST tree to preserve pragmas for further analysis by macros.
  • Custom pragmas are now supported for var and let symbols.
  • Tuple unpacking is now supported for constants and for loop variables.
  • Case object branches can be initialized with a runtime discriminator if possible discriminator values are constrained within a case statement.

Language changes

  • The standard extension for SCF (source code filters) files was changed from .tmpl to .nimf, it’s more recognizable and allows tools like Github to recognize it as Nim, see #9647. The previous extension will continue to work.

  • Pragma syntax is now consistent. Previous syntax where type pragmas did not follow the type name is now deprecated. Also pragma before generic parameter list is deprecated to be consistent with how pragmas are used with a proc. See #8514 and #1872 for further details.

  • Hash sets and tables are initialized by default. The explicit initHashSet, initTable, etc. are not needed anymore.

Tool changes

  • jsondoc now includes a moduleDescription field with the module description. jsondoc0 shows comments as its own objects as shown in the documentation.
  • nimpretty: –backup now defaults to off instead of on and the flag was undocumented; use git instead of relying on backup files.
  • koch now defaults to build the latest stable Nimble version unless you explicitly ask for the latest master version via --latest.

Compiler changes

  • The deprecated fmod proc is now unavailable on the VM.
  • A new --outdir option was added.
  • The compiled JavaScript file for the project produced by executing nim js will no longer be placed in the nimcache directory.
  • The --hotCodeReloading has been implemented for the native targets. The compiler also provides a new more flexible API for handling the hot code reloading events in the code.
  • The compiler now supports a --expandMacro:macroNameHere switch for easy introspection into what a macro expands into.
  • The -d:release switch now does not disable runtime checks anymore. For a release build that also disables runtime checks use -d:release -d:danger or simply -d:danger.

Bugfixes

  • Fixed “distinct generic typeclass not treated as distinct” (#4435)
  • Fixed “multiple dynlib pragmas with function calls conflict with each other causing link time error” (#9222)
  • Fixed “[RFC] extractFilename("usr/lib/") should return “lib” (not “”) and be called baseName (since works with dirs)” (#8341)
  • Fixed “\0 in comment replaced with 0 in docs” (#8841)
  • Fixed “round function in Math library sometimes doesn’t work” (#9082)
  • Fixed “Async readAll in httpclient produces garbled output.” (#8994)
  • Fixed “[regression] project config.nims not being read anymore” (#9264)
  • Fixed “Using iterator within another iterator fails” (#3819)
  • Fixed “nim js -o:dirname main.nim writes nothing, and no error shown” (#9154)
  • Fixed “Wrong number of deallocations when using destructors” (#9263)
  • Fixed “devel docs in nim-lang.github.io Source links point to master instead of devel” (#9295)
  • Fixed “compiler/nimeval can’t be used twice: fails 2nd time with: Error: internal error: n is not nil” (#9180)
  • Fixed “Codegen bug with exportc” (#9297)
  • Fixed “Regular Expressions: replacing empty patterns only works correctly in nre” (#9306)
  • Fixed “Openarray: internal compiler error when accessing length if not a param” (#9281)
  • Fixed “finish completely removing web folder” (#9304)
  • Fixed “counting the empty substring in a string results in infinite loop” (#8919)
  • Fixed “[Destructors] Wrong moves and copies” (#9294)
  • Fixed “proc isNil*(x: Any): bool = should be updated with non nil string, seq” (#8916)
  • Fixed “doAssert AST expansion excessive” (#8518)
  • Fixed “when Foo (of type iterator) is used where an expression is expected, show useful err msg instead of confusing Error: attempting to call undeclared routine Foo” (#8671)
  • Fixed “List comprehensions do not work with generic parameter” (#5707)
  • Fixed “strutils/isUpperAscii and unicode/isUpper consider space, punctuations, numbers as “lowercase”” (#7963)
  • Fixed “Regular Expressions: replacing empty patterns only works correctly in nre” (#9306)
  • Fixed “BUG: os.isHidden doesn’t work with directories; should use just paths, not filesystem access” (#8225)
  • Fixed “Unable to create distinct tuple in a const with a type declaration” (#2760)
  • Fixed “[nimpretty] raw strings are transformed into normal strings” (#9236)
  • Fixed “[nimpretty] proc is transfered to incorrect code” (#8626)
  • Fixed “[nimpretty] Additional new line is added with each format” (#9144)
  • Fixed ““%NIM%/config/nim.cfg” is not being read” (#9244)
  • Fixed “Illegal capture on async proc (except when the argument is seq)” (#2361)
  • Fixed “Jsondoc0 doesn’t output module comments.” (#9364)
  • Fixed “NimPretty has troubles with source code filter” (#9384)
  • Fixed “tfragment_gc test is flaky on OSX” (#9421)
  • Fixed “ansi color code templates fail to bind symbols” (#9394)
  • Fixed “Term write rule crash compiler.” (#7972)
  • Fixed “SIGSEGV when converting lines to closure iterator, most likely caused by defer” (#5321)
  • Fixed “SIGSEGV during the compile” (#5519)
  • Fixed “Compiler crash when creating a variant type” (#6220)
  • Fixed ““continue” inside a block without loops gives “SIGSEGV: Illegal storage access. (Attempt to read from nil?)”” (#6367)
  • Fixed “old changelogs should be kept instead of erased” (#9376)
  • Fixed “illegal recursion with generic typeclass” (#4674)
  • Fixed “Crash when closing an unopened file on debian 8.” (#9456)
  • Fixed “nimpretty joins regular and doc comment” (#9400)
  • Fixed “nimpretty changes indentation level of trailing comment” (#9398)
  • Fixed “Some bugs with nimpretty” (#8078)
  • Fixed “Computed gotos: bad codegen, label collision with if/statement in the while body” (#9276)
  • Fixed “nimpretty not idempotent: keeps adding newlines below block comment” (#9483)
  • Fixed “nimpretty shouldn’t format differently whether there’s a top-level newline” (#9484)
  • Fixed “Regression: 0.18 code with mapIt() fails to compile on 0.19” (#9093)
  • Fixed “nimpretty shouldn’t change file modif time if no changes => use os.updateFile” (#9499)
  • Fixed “Nim/compiler/pathutils.nim(226, 12) canon"/foo/../bar" == "/bar" [AssertionError]” (#9507)
  • Fixed “nimpretty adds a space before type, ptr, ref, object in wrong places” (#9504)
  • Fixed “nimpretty badly indents block comment” (#9500)
  • Fixed “typeof: Error: illformed AST: typeof(myIter(), typeOfIter)” (#9498)
  • Fixed “newAsyncSmtp() raises exception with Nim 0.19.0” (#9358)
  • Fixed “nimpretty wrongly adds empty newlines inside proc signature” (#9506)
  • Fixed “HttpClient: requesting URL with no scheme fails” (#7842)
  • Fixed “Duplicate definition in cpp codegen” (#6986)
  • Fixed “Sugar - distinctBase: undeclared identifier uncheckedArray” (#9532)
  • Fixed “Portable fsmonitor” (#6718)
  • Fixed “Small RFC. Minimal stacktrace for Exceptions when frames are disabled” (#9434)
  • Fixed “nim doc strutils.nim fails on 32 bit compiler with AssertionError on a RunnableExample” (#9525)
  • Fixed “Error: undeclared identifier: ‘|’” (#9540)
  • Fixed “using Selectors, Error: undeclared field: ‘OSErrorCode’” (#7667)
  • Fixed “The “–” template from module nimscript mis-translates “out” key” (#6011)
  • Fixed “logging error should go to stderr instead of stdout” (#9547)
  • Fixed “when in generic should fail earlier” (#8603)
  • Fixed “C++ codegen error when iterating in finally block in topmost scope” (#5549)
  • Fixed “document nim --nep1:on” (#9564)
  • Fixed “C++ codegen error when iterating in finally block in topmost scope” (#5549)
  • Fixed “strutils.multiReplace() crashes if search string is “”” (#9557)
  • Fixed “Missing docstrings are replaced with other text” (#9169)
  • Fixed “Type which followed by a function and generated by a template will not shown in docs generated by nim doc” (#9235)
  • Fixed “templates expand doc comments as documentation of other procedures” (#9432)
  • Fixed “please implement http put and delete in httpClient” (#8777)
  • Fixed “Module docs: 2 suggestions…” (#5525)
  • Fixed “math.hypot under/overflows” (#9585)
  • Fixed “=sink gets called on result when not used explicitly” (#9594)
  • Fixed “Treat compl as a c++ keyword” (#6836)
  • Fixed “Path in error message has ..\..\..\..\..\ prefix since 0.19.0” (#9556)
  • Fixed “nim check gives SIGSEGV: Illegal storage access ; maybe because of sizeof” (#9610)
  • Fixed “Cannot use a typedesc variable in a template” (#9611)
  • Fixed “=sink gets called on result when not used explicitly” (#9594)
  • Fixed “[NimScript] Error: arguments can only be given if the ‘–run’ option is selected” (#9246)
  • Fixed “macros.getTypeImpl regression, crash when trying to query type information from ref object” (#9600)
  • Fixed “[Regression] Complex.re and Complex.im are private” (#9639)
  • Fixed “nim check: internal error: (filename: "vmgen.nim", line: 1119, column: 19)” (#9609)
  • Fixed “optInd missing indent specification in grammar.txt” (#9608)
  • Fixed “not as prefix operator causes problems” (#9574)
  • Fixed “It is not possible to specify a pragma for the proc that returns lent T” (#9633)
  • Fixed “Compiler crash when initializing table with module name” (#9319)
  • Fixed “compiler crash” (#8335)
  • Fixed ““SIGSEGV” without any “undeclared identifier” error” (#8011)
  • Fixed “Incorrect parseopt parsing ?” (#9619)
  • Fixed “Operator or causes a future to be completed more than once” (#8982)
  • Fixed “Nimpretty adds instead of removes incorrect spacing inside backticks” (#9673)
  • Fixed “nimpretty should hardcode indentation amount to 2 spaces” (#9502)
  • Fixed “callSoon() is not working prior getGlobalDispatcher().” (#7192)
  • Fixed “use nimf as standardized extention for nim files with source code filter?” (#9647)
  • Fixed “Banning copy for a type prevents composing” (#9692)
  • Fixed “smtp module doesn’t support threads.” (#9728)
  • Fixed “Compiler segfault (stack overflow) compiling code on 0.19.0 that works on 0.18.0” (#9694)
  • Fixed “nre doesn’t document quantifiers re"foo{2,4}"” (#9470)
  • Fixed “ospaths still referenced despite its deprecation” (#9671)
  • Fixed “move on dereferenced pointer results in bogus value” (#9743)
  • Fixed “regression in discard statement” (#9726)
  • Fixed “try statements and exceptions do not cooperate well” (#96)
  • Fixed “XDeclaredButNotUsed doesn’t work with template, let/var/const, type; works with all other routine nodes” (#9764)
  • Fixed “ Warning: fun is deprecated doesn’t check whether deprecated overload is actually used” (#9759)
  • Fixed “Regression: tuple sizeof is incorrect if contains imported object” (#9794)
  • Fixed “Internal error when calling =destroy without declaration” (#9675)
  • Fixed “Internal error if =sink is used explictly” (#7365)
  • Fixed “unicode.strip behaving oddly” (#9800)
  • Fixed “X_examples.nim generated by runnableExamples should show line number where they came from” (#8289)
  • Fixed “quit() fails with “unreachable statement after ‘return’”” (#9832)
  • Fixed “quit() fails with “unreachable statement after ‘return’”” (#9832)
  • Fixed “Error: internal error: genLiteral: ty is nil when a var is accessed in quote do” (#9864)
  • Fixed “Regression: sizeof Error: cannot instantiate: ‘T’” (#9868)
  • Fixed “Using a template as a routine pragma no longer works” (#9614)
  • Fixed “Clang error on Rosencrantz” (#9441)
  • Fixed “Enum fields get hintXDeclaredButNotUsed hint even when marked with used pragma” (#9896)
  • Fixed “internal error: environment misses” (#9476)
  • Fixed “SIGSEGV: setLen on a seq doesn’t construct objects at CT” (#9872)
  • Fixed “Latest HEAD segfaults when compiling Aporia” (#9889)
  • Fixed “Unnecessary semicolon in error message” (#9907)
  • Fixed “koch temp c t.nim tries to look up t.nim in nim install directory (alongside koch)” (#9913)
  • Fixed “Regression: sizeof Error: cannot instantiate: ‘T’” (#9868)
  • Fixed “Showstopper regression: Nimscript no longer works “ (#9965)
  • Fixed “Global imports in cfg file broken” (#9978)
  • Fixed “Global imports in cfg file broken” (#9978)
  • Fixed “Regression - Nim compiler shows all gcc commands used when config.nims present” (#9982)
  • Fixed “[regression] Nimscript makes a program slower and more bloated” (#9995)
  • Fixed “Regression in Nimscript projectDir() behavior, returns empty string” (#9985)
  • Fixed “Global imports don’t work for non-std modules” (#9994)
  • Fixed “Object constructor regression in JS backend” (#10005)
  • Fixed “Regression: nimble install fails on nim devel” (#9991)
  • Fixed “Another config.nims regression” (#9989)
  • Fixed “nim js -d:nodejs main.nim gives: system.nim(1443, 7) Error: cannot 'importc' variable at compile time with a config.nims” (#9153)
  • Fixed “how to profile? using --profiler:on causes: Error: undeclared identifier: ‘framePtr’” (#8991)
  • Fixed “nim doc fail on lib/system/profiler.nim” (#9420)
  • Fixed “[regression] ./koch tests: Error: overloaded ‘readFile’ leads to ambiguous calls (with ~/.config/nim/config.nims)” (#9120)
  • Fixed “regression: normalizePath(“foo/..”) now incorrectly returns "", should be "." like before + in almost all other languages” (#10017)
  • Fixed “Incorrect ‘not all cases are covered’ when using enums with nonconsecutive items” (#3060)
  • Fixed “[ospaths] BUG: splitFile(“/a.txt”).dir = “” ; + other bugs with splitFile” (#8255)
  • Fixed “GC bug: seems very slow where it shouldn’t; maybe it leaks?” (#10040)
  • Fixed “Closure bug with the JS backend” (#7048)
  • Fixed “Error: unhandled exception: sym is not accessible [FieldError]” (#10058)
  • Fixed “with --errorMax:100 ; link step should not be attempted if previous step failed” (#9933)
  • Fixed “import os or ospaths compilation error in js” (#10066)
  • Fixed “Example for system.$[T: tuple | object] is misleading” (#7898)
  • Fixed “Combining object variants and inheritance leads to SIGSEGV during compilation” (#10033)
  • Fixed “Regression in distros.nim (foreignDep fails to compile)” (#10024)
  • Fixed “Testament megatest fails with Nim not found” (#10049)
  • Fixed “XDeclaredButNotUsed shows redundant info: declaration location shown twice” (#10101)
  • Fixed “Nim beginner’s feedback: “echo type(1)” does not work” (#5827)
  • Fixed “sizeof still broken with regard to bitsize/packed bitfields” (#10082)
  • Fixed “Codegen init regression” (#10148)
  • Fixed “toInt doesn’t raise an exception” (#2764)
  • Fixed “allow import inside block: makes N runnableExamples run N x faster, minimizes scope pollution” (#9300)
  • Fixed “Extra procs & docs for the unicode module” (#2353)
  • Fixed “regression: CI failing Error: unhandled exception: cannot open: /Users/travis/.cache/nim/docgen_sample_d/runnableExamples/docgen_sample_examples.nim [IOError]” (#10188)
  • Fixed “getAddrInfo index out of bounds error” (#10198)
  • Fixed “can’t build a tuple with static int element” (#10073)
  • Fixed “nimpretty creates foo.nim.backup for foo.nims” (#10211)
  • Fixed “regression caused by WEXITSTATUS: nim cpp compiler/nim.nim fails on OSX” (#10231)
  • Fixed “travis and appveyor should move the bulk of its logic to running a nim file” (#10041)
  • Fixed “Error: undeclared field: 'foo' should show type (+ where type is defined) (hard to guess in generic code)” (#8794)
  • Fixed “Discrepancy in Documentation About ‘f128 Type-Suffix” (#10213)
  • Fixed “Incorrect error message” (#10251)
  • Fixed “CI should call ./koch tools ; right now nimfind isn’t even being compiled” (#10039)
  • Fixed “Building koch from nim devel fails when config.nims importing os present” (#10030)
  • Fixed “unittest module uses programResult to report number of failures which can wrap” (#10261)
  • Fixed “Nimscript doesn’t raise any exceptions” (#10240)
  • Fixed “{.push raises: [].} breaks when combined with certain symbols” (#10216)
  • Fixed “Support “#.” for auto-enumerated lists in RST docs” (#8158)
  • Fixed “OpenSSL error breaking nimble and every package” (#10281)
  • Fixed “execShellCmd returns 0 instead of nonzero when child process exits with signal (eg SIGSEGV)” (#10273)
  • Fixed “nim check (and nim c –errorMax:0) SIGSEGV on first index out of bounds error” (#10104)
  • Fixed “Module db_sqlite doesn’t finalize statements with db_sqlite.rows after breaking the iterator’s loop” (#7241)
  • Fixed “Performance regression with –gc:markandsweep” (#10271)
  • Fixed “internal error when using typedesc is comparison in a macro” (#10136)
  • Fixed “cannot call template/macros with varargs[typed] to varargs[untyped]” (#10075)
  • Fixed “nim v0.13.0 breaks symbol lookup in quote block” (#3744)
  • Fixed “Some nimgrep issues” (#989)
  • Fixed “Safecall problem?” (#9218)
  • Fixed “Nim script is not supporting reading from stdin.” (#3983)
  • Fixed “Parameter constraints: undeclared identifier ‘{}’ within a template scope” (#7524)
  • Fixed “repr does not work with ‘var openarray’ parameter in function” (#7878)
  • Fixed “CountTable raisen error instead of returning a count of 0” (#10065)
  • Fixed “nim c -r main.nim foo1 "" foo3 doesn’t handle empty params or params w quotes” (#9842)
  • Fixed “refs #10249 ; more debug info to diagnose failures” (#10266)
  • Fixed “ObjectAssignmentError for aliased types” (#10203)
  • Fixed “nim cpp treats Nan as 0.0 (during compile time)” (#10305)
  • Fixed “terminal.nim colored output is not GCSAFE.” (#8294)
  • Fixed “Building koch from nim devel fails when config.nims importing os present” (#10030)
  • Fixed “every binary cmd line option should allow on/off switch” (#9629)

  • Fixed “Wrong bounds check using template [] to access array in a const object” (#3899)
  • Fixed “tdont_be_stupid.nim flaky test” (#10386)
  • Fixed “Separate nim install guide for users and packagers” (#5182)
  • Fixed “–embedsrc does not work on macos” (#10263)
  • Fixed “Devel regression on static semcheck” (#10339)
  • Fixed “vccexe.exe does not work without VS2015 x64 Native Tools command prompt.” (#10358)
  • Fixed “ospaths still referenced despite its deprecation” (#9671)
  • Fixed “Regression in sequtils” (#10433)
  • Fixed “Path in error message has ..\..\..\..\..\ prefix since 0.19.0” (#9556)
  • Fixed ““contributing” is listed as a module on theindex” (#10287)
  • Fixed “const Foo=int compiles; is that legal? what does it mean?” (#8610)
  • Fixed “parsecsv can’t handle empty lines at the beginning of the file” (#8365)
  • Fixed “Generated c code is not compile with the vcc cl.exe before 2012 after v0.19” (#10352)
  • Fixed “[Regression] converter to string leads fail to compile on 0.19” (#9149)
  • Fixed “regression: memory leak with default GC” (#10488)
  • Fixed “oids counter starts at zero; spec says it should be random” (#2796)
  • Fixed “re quantifier{ under-documented” (#9471)
  • Fixed “Minor issues in docs regarding keywords” (#9725)
  • Fixed “Explained the proc "pretty" in detail, file: json.nim with comments and sample program” (#10466)
  • Fixed “net.recvFrom address is always “0.0.0.0” for ipv6” (#7634)
  • Fixed “import “path with space/bar.nim” gives error msg with wrong file name” (#10042)
  • Fixed “Deprecation warnings for enum values print twice” (#8063)
  • Fixed “Undefined behaviour in the usage of incrSeqV3” (#10568)
  • Fixed “SetMaxPoolSize not heeded” (#10584)
  • Fixed “CI broken: tests/macros/t8997.nim fails” (#10591)
  • Fixed “prevent common user config to interfere with testament” (#10573)
  • Fixed “static: writeFile doesn’t work anymore since system refactorings” (#10585)
  • Fixed “export statement doesn’t support directories” (#6227)
  • Fixed “https://nim-lang.github.io/Nim/io.html gives 404” (#10586)
  • Fixed “Choosenim fails with “ambiguous call” in rst.nim” (#10602)
  • Fixed “Enable experimental feature with command line argument has no effect.” (#10606)
  • Fixed “Comparing function pointer with nil marks the proc as not gcsafe” (#6955)
  • Fixed “httpclient.timeout not exported” (#10357)
  • Fixed “nim check SIGSEGV (causing nimsuggest to fail too)” (#10547)
  • Fixed “index out of bounds errors should show index and bound” (#9880)
  • Fixed “Enable experimental feature with command line argument has no effect.” (#10606)
  • Fixed “Comparing function pointer with nil marks the proc as not gcsafe” (#6955)
  • Fixed “httpclient.timeout not exported” (#10357)
  • Fixed “nim check SIGSEGV (causing nimsuggest to fail too)” (#10547)
  • Fixed “certain seq manipulations fail when compiled to JS” (#10651)
  • Fixed “system.insert does not work with strings in VM” (#10561)
  • Fixed “Doc suggestion: include a link to theindex.html on …” (#5515)
  • Fixed “koch boot fails on windows with choosenim-installed nim: proxyexe.nim error” (#10659)
  • Fixed “getImpl on type symbol hides implementation” (#10702)
  • Fixed “Missing stdlib modules” (#8164)
  • Fixed “No “correct” way to declare inheritable ref object” (#10195)
  • Fixed “Line number missing in stdlib trace” (#6832)
  • Fixed “Better support for modifying XmlNodes” (#3797)
  • Fixed “No documentation of AsyncStreams” (#6383)
  • Fixed “set[ in proc definition crashes compiler” (#10678)
  • Fixed “net.bindAddr fails binding to all interfaces if address == "" for ipv6” (#7633)
  • Fixed “Tuple unpacking of for vars fails inside generic proc” (#10727)
  • Fixed “initSet should be called initHashSet” (#10730)
  • Fixed “inheritable placement not symmetric between object and ref object” (#3012)
  • Fixed “Alloc functions have side effects, makes it hard to use side effect tracking with destructors” (#9746)
  • Fixed “hashes:hash returns different values on Windows/Linux” (#10771)
  • Fixed “switch(“cpu”, “i386”) with –cc:vcc doesn’t work when it is written on *.nims” (#10387)
  • Fixed “async call now treated as non-gc safed call?” (#10795)
  • Fixed “{.borrow.} hangs compiler on non-distinct type (should produce an error or warning)” (#10791)
  • Fixed “DCE regression: modules can’t be eliminated” (#10703)
  • Fixed “Unsafeaddr rendered as addr in typed AST “ (#10807)
  • Fixed “Rendering of return statements in typed AST” (#10805)
  • Fixed “Assigning shallow string to a field makes a copy” (#10845)
  • Fixed “func keyword for proc types doesn’t imply noSideEffect” (#10838)
  • Fixed “SPAN.attachedType in toc should have no width” (#10857)
  • Fixed “[docgen] Generic type pragmas in wrong place” (#10792)
  • Fixed “os.joinPaths documentation is inaccurate; should reference uri.combine” (#10836)
  • Fixed ““invalid indentation” when assigning macro with code block to const” (#10861)
  • Fixed “Nim crashes with SIGABRT after getting into a replaceTypeVars infinite loop.” (#10884)
  • Fixed “Booleans Work Wrong in Compile-time” (#10886)
  • Fixed “C / CPP backends differ in argument evaluation order” (#8202)
  • Fixed “Change in syntax breaks valid code” (#10896)
  • Fixed “auto return type in macros causes internal error” (#10904)
  • Fixed “Nim string definition conflicts with other C/C++ instances” (#10907)
  • Fixed “nim check crash with invalid code, lowest priority” (#10930)
  • Fixed “nim check crash due to typing error, lowest priority” (#10934)
  • Fixed “Stacktrace displayed two times” (#10922)
  • Fixed “Cpp codegen regression. Showstopper” (#10948)
  • Fixed “lent T can return garbage” (#10942)
  • Fixed “Regression. atomicInc doesn’t compile with vcc and i386” (#10953)
  • Fixed “{.pure.} has no effect on objects” (#10721)
  • Fixed “nimpretty doesn’t put space around operators like a<b => a < b” (#10200)
  • Fixed “nimpretty messes alignment, after import statement” (#9811)
  • Fixed “Destructor regression for tuples unpacking” (#10940)
  • Fixed “Link error when a module defines a global variable and has no stacktrace” (#10943)
  • Fixed “std/json fails to escape most non-printables, breaking generation and parsing” (#10541)
  • Fixed “rst/markdown parser can’t handle extra parentheses after link” (#10475)
  • Fixed “Random: proc rand(x: HSlice) requires substraction” (#7698)
  • Fixed “Bug in setTerminate()” (#10765)
  • Fixed “ICE when using –newruntime with proc returning tuple” (#11004)
  • Fixed “terminal.nim does not compile using –newruntime” (#11005)
  • Fixed “Casting a seq to another seq generates invalid code with –newruntime” (#11018)
  • Fixed “strformat/fmt doesn’t work for custom types [regression]” (#11012)
  • Fixed “Casting a seq to another seq generates invalid code with –newruntime” (#11018)
  • Fixed “newruntime - t.destructor != nil [AssertionError] with toTable()” (#11014)
  • Fixed “templates (e.g. sequtils.toSeq) often shadow result” (#10732)
  • Fixed “newruntime: Error: system module needs: NimStringDesc when calling $ inside method on an object variant” (#11048)
  • Fixed “newruntime: internal error when iterating over seq (which is a field of an object) inside methods” (#11050)
  • Fixed “Error: internal error: ‘=destroy’ operator not found for type owned Node” (#11053)
  • Fixed “new output can be assigned to an unowned ref” (#11073)
  • Fixed “Illegal storage access when adding to a ref seq” (#11065)
  • Fixed “strformat float printing doesn’t print “.0” portion [regression]” (#11089)
  • Fixed “nim doc2 ignores –docSeeSrcUrl parameter” (#6071)
  • Fixed “runnableExamples + forLoop = segfault” (#11078)
  • Fixed “destructible context without ‘result’ or ‘return’ should also be supported” (#1192)
  • Fixed “new Obj crashes at the end of the program on newruntime” (#11082)
  • Fixed “Documentation of the modules broken out of system.nim are missing “ (#10972)
  • Fixed “DFA regression. Branches of AST trees are missed in control flow graph.” (#11095)
  • Fixed “[Regression] nkIdentDefs can be left in vmgen” (#11111)
  • Fixed “JS target does not prevent calling compileTime procs” (#11133)
  • Fixed “rand can return invalid values of a range type” (#11015)
  • Fixed “compiler crash on discard void” (#7470)
  • Fixed “Unowned ref can trivially escape without causing any crashes” (#11114)
  • Fixed “Destructor lifting regression” (#11149)
  • Fixed “const alias to compile time function fails.” (#11045)
  • Fixed “Using type instead of typedesc in template signature fails compilation” (#11058)
  • Fixed “Compiler error caused by quote do: else” (#11175)
  • Fixed “cast to non ptr UncheckedArray does not produce an error or warning” (#9403)
  • Fixed “openArray generates incorrect C code with “incomplete type”” (#9578)
  • Fixed “os:standalone Error: system module needs: appendString” (#10978)
  • Fixed “gensym regression” (#10192)
  • Fixed “new: module names need to be unique per Nimble broken on Windows” (#11196)
  • Fixed “Compiler crash on cfsml bindings” (#11200)
  • Fixed “Newruntime: compileTime variables can cause compilation to fail due to destructor injections” (#11204)
  • Fixed “object self-assignment order-of-evaluation broken” (#9844)
  • Fixed “seq self-assignment order-of-evaluation broken” (#9684)
  • Fixed “Compiler crash with generic types and static generic parameters” (#7569)
  • Fixed “C macro identifiers (e.g. errno) are not properly avoided in code generation” (#11153)
  • Fixed “SIGSEGV in asgnRefNoCycle with const sequence” (#9825)
  • Fixed “asyncdispatch could not be linked to nimrtl” (#6855)
  • Fixed “Newruntime: Bad C++ codegen for ref types with destructors” (#11215)
  • Fixed “Better error message for object variant with enum that is below it” (#4140)
  • Fixed “Can’t declare a mixin.” (#11237)
  • Fixed “Nim doc mangles signed octal literals” (#11131)
  • Fixed “Selectors, Error: undeclared field: ‘OSErrorCode’ on macOS” (#11124)
  • Fixed “--cppCompileToNamespace:foo fails compilation with import os” (#11194)
  • Fixed “[OpenMP] Nim symbol interpolation support” (#9365)
  • Fixed “Inconsistent typing error with gensymed var” (#7937)
  • Fixed “New module names break file-specific flags” (#11202)
  • Fixed “inheritance for generics does not work” (#88)
  • Fixed “Possible bug related to generics type resolution/matched” (#6732)
  • Fixed “static range type bounds not checked when conversion from intLit” (#3766)
  • Fixed “threadpool: sync() deadlocks in high-CPU-usage scenarios” (#11250)
  • Fixed “var result array - bad codegen (null pointer dereference)” (#8053)
  • Fixed “future/sugar => syntax breaks with generics” (#7816)
  • Fixed “os.joinPath removes the leading backslash from UNC paths (regression)” (#10952)
  • Fixed “re: memory leak when calling re proc repeatedly” (#11139)
  • Fixed “threadpool: tests/parallel/tconvexhull.nim segfaults intermittently on systems with more than 4 cores” (#11275)
  • Fixed “Not equal when streams.readBool and peekBool compare true” (#11049)
  • Fixed “os.tailDir fails on some paths” (#8395)
  • Fixed “Power op ^ is not optimized for a: int; echo a ^ 2 case (minor bug)” (#10910)
  • Fixed “str &= data doesn’t behave as str = str & data.” (#10963)
  • Fixed “Unable to make a const instance of an inherited, generic object.” (#11268)
  • Fixed “Overload precedence for non-builtin types” (#11239)
  • Fixed “Error when trying to iterate a distinct type based array” (#7167)
  • Fixed “Objects marked with {.exportc.} should be fully defined in generated header” (#4723)
  • Fixed “Generic function specialization regression” (#6076)
  • Fixed “compiler should give ambiguity errors in case of multiple compatible matches” (#8568)
  • Fixed “xmltree.findAll doesn’t work as expected with htmlparser for non-lowercase names” (#8329)
  • Fixed “wrong stack trace information about the raised exception” (#11309)
  • Fixed “Newruntime: owned procs don’t implicitly convert to unowned in ==” (#11257)
  • Fixed “order of imports can cause errors” (#11187)
  • Fixed “Passing code via stdin to Nim stopped working [regression Nim 0.19+]” (#11294)
  • Fixed “”–out:” switch is ineffective with “nim doc” [regression]” (#11312)
  • Fixed “VC++ broken in devel: module machine type ‘X86’ conflicts with target machine type ‘x64’” (#11306)
  • Fixed “Code that used multi aspect of multimethod now crashes at runtime” (#10912)
  • Fixed “symbol resolution issues when ambiguous call happens in generic proc” (#11188)
  • Fixed “compile pragma name collision” (#10299)
  • Fixed “Unexpected behaviour on method dispatch” (#10038)
  • Fixed “Nim object variant issue” (#1286)
  • Fixed “json.to macro cannot handle ambiguous types even in a full form (module.Type)” (#11057)
  • Fixed “Out of bounds access in CritBitTree” (#11344)
  • Fixed “Newruntime: assignment to discriminant field in case objects not supported” (#11205)
  • Fixed “Dynamic dispatch broken if base method returns generic var type” (#6777)
  • Fixed “newruntime and unused generics: compiler crash” (#6755)
  • Fixed “Type aliases do not work with Exceptions.” (#10889)
  • Fixed “Compiler crash when accessing constant in nested template” (#5235)
  • Fixed “unicode.nim Error: type mismatch: got <seq[char]> but expected ‘string’” (#9762)
  • Fixed “Internal error with auto return in closure iterator” (#5859)
  • Fixed “[Compiler Crash] - getAST + hasCustomPragma” (#7615)
  • Fixed “debug mode compiler crash when executing some compile time code” (#8199)
  • Fixed “Compiler does not set .typ inside macros when creating literal NimNodes” (#7792)
  • Fixed “Error: internal error: expr: var not init sevColor_994035” (#8573)
  • Fixed “internal error: could not find env param for when one iterator references another” (#9827)
  • Fixed “internal error when assigning a type to a constant of typedesc” (#9961)
  • Fixed “Overload resolution regression” (#11375)
  • Fixed “strutils: toBin(64) uses ‘/’ for the 63rd bit if it’s set” (#11369)
  • Fixed “base64.encode should not “prettify” the result by default” (#11364)
  • Fixed “Nim ships latest nimble rather than stable” (#11402)
  • Fixed “debugger:native no longer generates pdb file with cc:vcc” (#11405)

HookraceChinese Traffic to time.gif

Nearly two years ago I posted this endless GIF that always shows the current time in UTC:

time.gif (reload if it fails)

Now looking at my GoAccess dashboard I can see that it is picking up in popularity rather suddenly:

GoAccess excerpt

But strangely I can’t find anything about time.gif being linked on the web. So this might just be an attempted Denial of Service (DoS) attack? At least that would be something I am familiar with from the DDNet direction, but it’s certainly strange on HookRace. But instead of simply shutting down time.gif I decided to try to find out who is accessing it and whether I can keep the server up.

Let’s look into the nginx logs, since I use nginx to proxy the requests to the Haskell program. There I see about 40 new requests per second looking like this:

hookrace.net 123.185.XXX.XXX - - [21/May/2019:21:21:27 +0200] "GET /time.gif HTTP/2.0" 200 3335 "XXX" "Mozilla/5.0 (Linux; Android 8.1.0; V1818A Build/OPM1.171019.026; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/66.0.3359.126 MQQBrowser/6.2 TBS/044681 Mobile Safari/537.36 MMWEBID/XXX MicroMessenger/7.0.4.1420(0x2700043A) Process/tools NetType/WIFI Language/zh_CN" 8.055
hookrace.net 111.62.XXX.XXX - - [21/May/2019:21:21:27 +0200] "GET /time.gif HTTP/2.0" 200 32061 "XXX" "Mozilla/5.0 (Linux; Android 5.1; OPPO A59s Build/LMY47I; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/66.0.3359.126 MQQBrowser/6.2 TBS/044704 Mobile Safari/537.36 MMWEBID/XXX MicroMessenger/7.0.4.1420(0x2700043B) Process/tools NetType/WIFI Language/zh_CN" 89.256
hookrace.net 111.29.XXX.XXX - - [21/May/2019:21:21:27 +0200] "GET /time.gif HTTP/2.0" 200 543830 "XXX" "Mozilla/5.0 (Linux; Android 7.1.1; OPPO R11 Build/NMF26X; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/66.0.3359.126 MQQBrowser/6.2 TBS/044704 Mobile Safari/537.36 MMWEBID/XXX MicroMessenger/7.0.4.1420(0x2700043A) Process/tools NetType/WIFI Language/zh_CN" 1540.238
hookrace.net 112.2.XXX.XXX - - [21/May/2019:21:21:27 +0200] "GET /time.gif HTTP/2.0" 200 172102 "XXX" "Mozilla/5.0 (Linux; Android 8.1.0; V1816A Build/OPM1.171019.011; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/66.0.3359.126 MQQBrowser/6.2 TBS/044611 Mobile Safari/537.36 MMWEBID/XXX MicroMessenger/7.0.4.1420(0x2700043A) Process/tools NetType/WIFI Language/zh_CN" 492.600
hookrace.net 123.13.XXX.XXX - - [21/May/2019:21:21:27 +0200] "GET /time.gif HTTP/2.0" 200 1275 "XXX" "Mozilla/5.0 (Linux; Android 9; LYA-AL00 Build/HUAWEILYA-AL00L; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/66.0.3359.126 MQQBrowser/6.2 TBS/044704 Mobile Safari/537.36 MMWEBID/XXX MicroMessenger/7.0.4.1420(0x2700043A) Process/tools NetType/WIFI Language/zh_CN" 1.888
hookrace.net 117.91.XXX.XXX - - [21/May/2019:21:21:27 +0200] "GET /time.gif HTTP/2.0" 200 4684 "XXX" "Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16D57 MicroMessenger/7.0.3(0x17000321) NetType/WIFI Language/zh_CN" 12.123

I checked a few IP addresses and they were all in mobile networks, not data centers. The user agent containing MicroMessenger and MQQBrowser indicates that the source of the traffic are WeChat and/or QQ, popular chinese chat apps.

Quantifying the Traffic

For reference, the system I’m running on is a simple Debian based VPS with 2 threads and 2 GB of RAM that also functions as the main server for DDNet’s website, database and my HookRace blog.

I already had to do some scaling when posting the initial blog post on Hacker News, optimizing the Haskell application itself to use LZW encoding in the GIF frames, to properly clean up connections to prevent any memory leaks and disable buffering in nginx’s config.

But the current level of traffic is on a different scale with 2.4 million hits on time.gif in the last 23 hours (30 hits per second) resulting in 113 GB of data being transferred. And many of those connections don’t finish quickly, instead they linger for seconds, minutes or even hours.

Using lsof -i | grep Time | wc -l I can see that there are about 6000 people downloading the GIF at peak times, causing up to 30 Mbit/s of outgoing traffic with 7000 packets/second incoming and the same number outgoing. The DDNet server statistics lets me monitor this nicely (related blog article):

Network Network Traffic Packets Network Packets CPU CPU Usage

Keeping Up with the Traffic

Regenerating the ranks pages of DDNet usually causes the main CPU load on the server, which can be seen in the above CPU graph as spikes. This task is already set to only run when the server is below a specified load, so that more essential tasks have priority.

The first new problem was nginx running into a limit of 768 worker_connections:

2019/05/20 20:41:30 [alert] 761#761: *3828093 768 worker_connections are not enough while connecting to upstream, client: 49.114.XXX.XXX, server: hookrace.net, request: "GET /time.gif HTTP/2.0", upstream: "http://127.0.0.1:5002/", host: "hookrace.net", referrer: "XXX"

Luckily that is easily fixed in /etc/nginx/nginx.conf by increasing the number of worker_connections to keep alive, each of which is handling one of the long-lasting time.gif requests:

events {
  worker_connections 20000;
}

and systemctl reload nginx. No downtime required since nginx will start new worker processes to handle new requests while keeping the old ones alive for a time to keep handling existing connections.

Unfortunately that fix only lasted a few hours until the next problem appeared:

2019/05/20 23:09:21 [alert] 15188#15188: *4041619 socket() failed (24: Too many open files) while connecting to upstream, client: 27.207.XXX.XXX, server: hookrace.net, request: "GET /time.gif HTTP/2.0", upstream: "http://127.0.0.1:5002/", host: "hookrace.net", referrer: "XXX"

Increasing the limits in /etc/security/limits.conf for the nginx user fixes this:

#<domain>      <type>  <item>         <value>
nginx          soft    nofile         1048576
nginx          hard    nofile         1048576

The value of 1048576 is chosen since it’s the value set in sysctl fs.file-max and it should be good enough for now.

Next I noticed that the server was running out of memory with both the Haskell application and nginx having to keep track of so many connections at once. For now I increased the swap size on the fly to keep some less commonly used stuff there using dd if=/dev/zero of=/var/swap bs=1M count=5000 && mkswap /var/swap && swapon /var/swap.

When running out of memory I noticed that Python’s msgpack implementation fails quite confusingly when it runs OOM. So I had to add some fixes to the code creating the DDNet ranks pages to handle this possibility.

The Linux Kernel’s TCP buffers ran out of memory next, complaining in dmesg:

[1638211.984805] TCP: out of memory -- consider tuning tcp_mem

So I increased them with a net.ipv4.tcp_mem = 116730 155640 233460 in /etc/sysctl.conf and reloaded it with sysctl -p.

A limitation of my current approach is the number of ports nginx can open to proxy to the Haskell application. If that gets blown I’ll have to communicate to the application differently or simply redirect to the application directly instead of proxying it. That would also reduce the CPU load significantly, cutting out nginx which happens to be much more expensive than the Haskell application, probably because it’s also handling TLS.

Final Words

While it was fun to keep time.gif running in the face of this amount of traffic, I still haven’t answered the final question of where this traffic is coming from. It might be that lots of Chinese happen to be spreading time.gif on WeChat and QQ, but for that the traffic looks a bit too sterile. Has anyone seen similar traffic patterns and might know if they are real or some kind of botnet? Maybe someone has embedded traffic.gif on some WeChat-specific page. If anyone has a clue please drop me an email at dennis@felsin9.de.

The best guess so far is by Kolen:

Hi,

I read your post and this is just my guess:

Chinese “WhatsApp” type of communication culture is very strange. They are like spam emails in the old days. Often time some one might create posts in picture, eg include warm message such as reminding each other to put on more clothes as the weather is getting cold, etc.

My guess is then someone read your trick and thought that it is a good idea to create some picture that always show the current time. Eg to remind others that it is time to sleep or something.

And like email spams in the old days, people can share things like this crazily, often by older people who don’t know much about technology. (Just like how some tweet can go viral, messages like this could also go viral within there networks. And unfortunately the viral thing we often received are really rubbish like.)

yours,

Kolen

Discuss on Hacker News.

Footnotes