>> "We observed a 5–35% speedup across our benchmarks."
Decreased compile times and binary size:
>> "While these changes across the compiler toolchain are mostly invisible, users have observed a significant speedup in compile time and a reduction in binary size by as much as 20–30%."
Vendoring dependencies by default:
>> "...and in Go 1.7... the "vendor" behavior is always enabled"
Context package added to std lib:
>> "To make use of contexts within the standard library and to encourage more extensive use, the package has been moved from the x/net repository to the standard library as the context package."
On our code base, we're seeing >2x build speed increases vs. go 1.6! Compiling the entire go monorepo, including vendored dependencies, dropped from 73 seconds to 29 on 4 CPUs when moving from 1.6.3->1.7rc6. Huge improvements to CI times, when amortized across go build, go test, docker builds etc.
Yes, compile times have gotten worse in some releases, but there were good technical reasons for the trade-off, mainly the general quality improvements of the go environment. But the good news is, they are not resting but trying to gain back the lost speed - you do not observe this always in larger programs.
Calling people happy about this "g-sheep" is unnecessary name-calling.
Following the twitter link and got this: "Something is technically wrong. Thanks for noticing—we're going to fix it up and have things back to normal soon."
I just learned Go 2 days ago and today I am already running my own web app!
I love Go because it is kind of like C, but better (more modern).Generally, I always program my stuff in Java, but whenever I have an idea of creating something I could only choose between: C, Java, bash. Obviously I am not going to use C and bash for most of my ideas. Thinking about solving my issues in Java is meh so often I decided it is not worth it to invest time. I feel like using Java for my ideas is like using a semi truck for roadtrips. It can be done, but it's just not very efficient having to launch the JVM everytime I want to do something small.
Yes, I was open to new languages, but I did not really care about: C++, Python, Perl, Ruby and so on because I never cared about web dev. Now I was bored and finally decided to learn Go.
I suggest giving Ruby a try too. I find it great for both web dev and local automation scripts. You can do pretty much anything you can do in bash but with clearer and shorter code. I've been writing all my non-trivial automation code in Ruby for years.
Rails is just a popular web framework for Ruby, there's a lot more the language can do.
This is a hurdle, but IMO it's not too annoying if you already have rights to install packages. Just automate interpreter installation in bash if it's not already there. Way too much time and money is invested in fighting the esoteric crannies of Bourne shell when the logic would be much clearer in Ruby or Python.
Ansible's moment in the sun is fading now that Docker came onto the scene, but that was a glorious moment. It's still very useful for anyone with long-running, non-removable machines. Ansible requires Python and allows a great deal of behavior to be defined in simple YAML instructions.
You might have the right to install packages, but that may not solve your problem, that the application may require different version than available in the repo.
The classical case is Ruby + Rails + Passenger + CentOS. While there is SCL with any Ruby and Rails version you might want, that does not mean, that the Passenger package will work with it :(.
Yeah, if you can install packages, then you can run the RVM installation script and install the right version of Ruby. In fact, you don't even need admin privileges if you use RVM.
That will get you off package system for the entire stack and now the maintenance of the whole circus is up to you. Which is exactly what I'm trying to get away from.
I do dislike individual language package managers and ecosystems for this reason, but I think it's gotta be accepted at this point. Practically every language has their own package repo and versioning apparatus now. Ruby has RVM, Node.js has NVM, Python has PyPi/venv, etc. Most projects expect their users to deploy with these and include things like Gemfiles to define dependencies. That usually means if you insist on using the upstream packages provided by the distribution, you're going to have to install them manually and have a bunch of extra headaches to worry about anyway (to be honest, most custom applications are really difficult to get running using only distro-packaged Ruby or Python libs).
The individual language package managers are very nice for development, as you may have any package version you want, and also for non-linux users, where there may be no native package management at all. For operations, however, they suck mightly (that's terminus technicus).
I quite like the SCL [1]. It solves the problem, that you might need different version of node/python/ruby/rails/php/whatever, than is packaged with the OS release of your choice. It has the software nicely packaged and maintained. Now getting third party packages to work with that, that's the problem. Passenger rpm [2] requires system-provided ruby, not just any ruby... I will survive that that App X wants Gem Y installed by rubygem. I won't like it, but it is workable and makes maintenance/updates slightly more difficutlt. But I will not put unpackaged module into Apache config.
Maybe try Crystal (https://crystal-lang.org/) then. It has many of the benefits of Go and a Ruby-like syntax. It feels even more like a scripting language than Go because of awesome type inference, uses a similar model as Go for concurrency, and even has proper generics ;-)
Agreed. I recently transitioned from Rails development to devops, and there's no Ruby on any system by default. My Bash got better fast!
Fortunately Elixir/Erlang releases include all the executables you'll need. They aren't quite as slick as Go binaries, which all live in one file, but it's still pretty cool.
If you like Go, I don't see much reason to try Ruby except intellectual curiosity. Ruby is clearly a language on its way out and it is not just a lot slower than Go, it's also dynamically typed.
Why is it 'clearly' on its way out? IMO Rails is still the best web framework around, and the last release has really put it ahead of its closest competition.
I can't speak to this. I'm a Go and Python programmer. I learned Go in about ~8 hours and have been very productive with it over the past 2 years.
I've been struggling at work to learn and use Rails for the better part of a year. I constantly have to grab someone and have them help me. Either I'm an idiot, or Rails has a bigger learning curve than people make it out to have because I've never experienced this in all my 10 years of programming professionally.
Ruby is too forgiving, has multiple syntaxes for the same thing, and lets people do really obscure shit in a large codebase (monkey patching, hidden imports, etc). In short: it enables developer laziness.
I've found Ruby code the hardest to debug over the years. Also, I think the web development community is waking up to the value of type safety.
I will use Ruby for the occasional script because I like its backtick syntax for invoking other commands, but that's it.
I'd second this. Rails is still probably the fastest way to get an MVP off the ground, and most of the time it's more than fast enough. You can get UX on par with a well done SPA almost entirely using server code with Turbolinks 5 + Action Cable, too.
Yes, this is also usually true. But if you don't know tools that are particularly well suited for the job, or you just don't like them and are looking for something else to try, I would put Rails at the top of the list for productivity. (For most use cases)
The only problem is that you need to maintain and enhance that MVP for the next several years -- unless it's just a hackathon.
I've spent way too much of my life trying to optimize Ruby performance and memory usage, switching from one web server to the next, then fixing threading issues in dependencies, and so on. It's not fun. All that upfront productivity is lost in the end.
Quite happy to spend more time upfront investing in a reliable end-product.
Agreed on popularity but Kickstarter is slow and memory is still expensive. Shorter code isn't always a good idea when most Rail and Ruby developers can't tell between APIs.
Swift is by far being the best all-rounder language and between the fastest and slowest.
After the transition from C to Go of the gc toolchain there was a lot of potential in both areas. The SSA backend increased even more that potential. And there is still room for improvement, so future versions will be even better. But yes, it's a great release. Congrats to the team!
BTW, what does one have to do to get fully statically linked executables? The binaries I've created with go get SOMETHING are linking against libc, but I remember that go doesn't rely on the system libc. Has that changed? For instance I just installed git-appraise, and it's a dynamically linked executable. What am I doing wrong?
Go generally links to libc to use the system's getaddrinfo-- some OSes override it to do special things with dynamically discovered hosts on local networks, interfacing with LDAP/AD, etc.
If you use `CGO_ENABLED=0 go build`, it should make a statically linked executable.
I don't believe that has been true for a few releases now. I believe go defaults to a pure Go resolver implementation now. You have to force enable the cgo resolver.
"When cgo is available, the cgo-based resolver is used instead under a variety of conditions: on systems that do not let programs make direct DNS requests (OS X), when the LOCALDOMAIN environment variable is present (even if empty), when the RES_OPTIONS or HOSTALIASES environment variable is non-empty, when the ASR_CONFIG environment variable is non-empty (OpenBSD only), when /etc/resolv.conf or /etc/nsswitch.conf specify the use of features that the Go resolver does not implement, and when the name being looked up ends in .local or is an mDNS name."
It is probably the net package. The net package depends on cgo and libc for some DNS resolution. There is a pure Go implementation, but the cgo one is used if cgo is available. If you disable cgo as the other comments say, the pure Go implementation will be used instead. Alternatively, you can not include the net package.
env CGO_ENABLED=0 go get github.com/google/git-appraise/git-appraise
but I always get an error that go wants to write net.a in /usr/lib, which I naturally am not allowed to do because it's owned by the distro (Arch) go package:
go install net: open /usr/lib/go/pkg/linux_amd64/net.a: permission denied
I have only set GOPATH to ~/.go and nothing else. Any other GO environment variable I should have used?
Yay, s390x support is finally in mainline! Finally we (SUSE) no longer needs to use gcc-go to build Go binaries on some platforms (we've had nothing but issues from gcc-go, half of the patches we apply to Docker are to make it behave when built with SLE's gcc version).
Ok, off topic question. Not an expert in grammar, but something about "Go 1.7 is released" seems wrong to me.
Anyone could tell me if this is an correct usage of grammar?
Edit: Dont know why so many downvote, but it is a honest question.
It seems right to me (American speaker). "Released" functions as an adjective, with similar meaning to "available." "Go 1.7 is available" would be grammatically correct.
"Go 1.7 has been released" would also be correct, but uses the present perfect tense.
So the question is, can "released" be used as an adjective? "The released version has a bug" seems correct to me, and "released" clearly functions as an adjective there.
All that said, the present perfect version seems clearly correct, while I can't make a great argument for the adjectival version.
It's just present tense, usually known as "historical present" tense when used in a headline like this. "Released" here might be intended as an adjective or verb; it's ambiguous, but both are valid in the sentence structure.
I am not a native English speaker myself. But yes that fragment while correct parsed a bit weirdly in my head because "is" is often used for properties of an entity like "John is dead" and "has been" is often used for something that is done to something. Like "John has been convicted". I think it has something to do with "is" being a simple present form while "released" in one sense talks about the state of the object in which case it is fine but also in another case is the action of getting released in which case it is in the simple past tense.
Like another commenter mentioned, this style is commonly used in headlines. It's referred to as historical present tense. Here are a few resources for more information:
"Is" combined with "released" does sound odd, but it's not too unusual, like in this sentence: "Heat is released into the atmosphere by cars." (From http://www.merriam-webster.com/dictionary/release)
Sometimes good advice, but certainly not a rule. As Pinker points out in "A Sense of Style", passive voice lets you modify word order to emphasize what's important to the reader.
Exactly, both would have been a fine. I am wondering if this is a grammar mistake, or general English usage difference in US and Commonwealth countries.
Isn't it the passive voice? You conjugate to be and add the past participle. "Go 1.7 is released" rather than "we have released Go 1.7". The passive voice makes Go the subject of the sentence, otherwise it would focus on who is doing the releasing which is less important in this context.
In informal writing you should avoid using the passive voice, as it sounds disconnected and boring. Technical articles can sound better in the passive voice though, and I remember my physics teachers always used to berate people for not using it. So we should have written "the experiment is performed" rather than "I performed the experiment".
>A new compiler back end, based on static single-assignment form (SSA), has been under development for the past year
Huh, I was under the impression that either SSA or CPS was pretty standard for any serious compiler. Does anyone know why they didn't design it for this from the beginning? It's like one of the earlier things you learn when making actual compilers.
It was more expedient to start with the Plan 9 tool chain that we had at the time. Ken was already familiar with the code base, which was small and compiled quickly (a few seconds). Because of this we got a lot done quickly, knowing that we would eventually modernize the compiler.
SSA is standard for the middle end but not for the backend. According to my knowledge there are only two compilers with an SSA-based backend: libFirm (www.libfirm.org) and the Go compiler.
Obviously no backend is in SSA format after register allocation, but before allocation what do other compilers use if not SSA? Is there some kind of low-level LLVM and GCC IR separate from the front-end and middle one? That isn't in SSA?
I know that the C2 and Graal compilers are SSA all the way until register allocation is done.
You can stay in SSA form even after register allocation. In libFirm the assigned registers are just attributes of the values in the SSA representation. There was some additional discussion regarding this in [1].
GCC uses a separate representation (RTL) for their backend that is not in SSA form [2]. LLVM stays in SSA form for some backend phases but lowers the SSA form before register allocation (as also mentioned in [1]).
I took a quick look at the C2 and the Graal compiler. C2 seems to use LLVM's code generator (and thus lowers SSA before register allocation) but Graal seems to support SSA-based register allocation, nice.
> SystemCertPool returns an error on Windows. Maybe it's fixable later
This doesn't inspire confidence in Go as a cross platform language which is at version 1.7. If this is implemented in say 1.8, am I supposed to check for Go versions in order to know if the SystemCertPool func works or not? I mean why not just release it when it works on all tier-1 supported platforms?
We actually try hard not to do platform-specific stuff, but in this case it seemed worth it, considering the problems people were having with cert validations and how Windows differs in how it validates TLS connections.
Go is far better than most languages at supporting random system APIs. I mean, you can cross-compile code on Windows that uses Linux syscalls. Try doing that in Python. Actually I just Googled and I'm not sure Python even lets you use syscalls. I also couldn't even find any third party library to give you the system CA pool, let alone a first party one.
Tell me a language that has a first-party cross-platform library that allows you to get the system CA pool on all platforms.
vendor directories are no longer optional, which is nice.
Go get now update submodules... I'm honestly not sure what the current hack is for package management, but I assume people are still doing wrappers around go get to pin to commits/versions, (or else building your own repos for funsies), and I'm wondering if that breaks anything.
There are active discussions on the Go Package Management mailing list and the intention to propose further improvements to the go toolchain for future releases. We had good discussions with the Go Team during GopherCon last month.
SSA for all architectures, then delete the old compiler backend and optimize the frontend which can then assume an SSA backend and do less work. (Currently the frontend assumes a less-capable backend so has to do more work, which actually makes SSA slower)
That's been my feeling for a long time now. After getting spoiled by Pythons enormous standard library, other languages feel incredibly bare-bones and getting things done takes a lot more work. I wish other languages focused more on including lots of library functions.
Size alone is only half (at best) of the equation. What makes Python's standard library stand out is the way it is structured and documented.
Consider Java's standard library, which is huge. That last few times I did anything in Java, I spent more time browsing the library documentation than coding. Admittedly, I did not know the library very well, if one uses Java all the time, that ratio probably changes. In Python I don't need to know the standard library well, because the way the documentation is written makes it very easy to find what I am looking for.
The one thing that really pisses me off about the way Go handled vendoring is that they did it in a way that makes it incompatible with GOPATH. Previously in runC and Docker, we had build hacks that would symlink (or full copy) the current directory into vendor/src/<package> and then set the GOPATH to vendor/. This was compatible with every go version. In addition, many other projects did the exact same thing.
But the way that vendoring works in Go 1.5 and up is that you make vendor not a valid GOPATH and you have to now either create a fake GOPATH and move your current directory into it, or you have to do some symlink stuff within vendor/ that doesn't really work. Why was such a small cosmetic change seen as a good idea? It's needlessly incompatible with previous ways of making vendoring work seamlessly with Go.
I'm hoping that the packaging discussions that are going to be happening over the next few months don't result in a similar decision that "we know best".
If anything, the Go people have repeatedly claimed they don't know best -- because Google famously uses a huge monorepo with all deps vendored, and don't do "package management" as such -- and have preferred to let the community find a solution organically.
Which is also why we ended up in this frustrating profusion of competing tools (Godeps, Glide and many others, plus the messy business that is git submodules) rather than a concerted effort towards a unified solution.
I don't at all believe the argument that they can't design something because they don't need it at Google. They're engineers, after all. You don't need to use a bridge in order to build one. There are plenty of existing tools (Bundler, Cargo, NPM, Cabal) that can also be used as case studies in how to do things right (Bundler, Cargo) or don't do them wrong (NPM).
In my company we use Glide, which is the best solution I've encountered so far, but it's beyond buggy, and struggles with some Google repos that need to be flattened in order to avoid duplicate, conflicting dependencies (glog and pflags, sigh).
> If anything, the Go people have repeatedly claimed they don't know best
So why is the defacto vendoring system for the Go tooling incompatible with old Go versions (the "organic" solutions were compatible with Go versions without vendoring support because they were implemented before vendoring support was a thing!)?
The vendor/src hack meant that you could compile with ANY version of Go -- that's important for distributions (and other people who want to build runC and Docker -- put plainly we pin Go versions because of compiler bugs). But no, now we have to use vendor/ which means that now we have to do even more messing around with the GOPATH (which makes rpmbuild even more ugly than it needs to be). sigh
I can't explain that since I'm not on the Go team. However, it's possible that they also didn't realize this hack was being relied on by people. The fact that you describe it as a hack implies it was never intentionally supported behaviour. Did you bring it up during the development of 1.5?
> The fact that you describe it as a hack implies it was never intentionally supported behaviour.
It's used by the largest Go projects that exist. So if the Go developers didn't look at what the largest Go projects do and take inspiration from that, I'm really not sure what else to say.
A hack that is used by everyone is relied upon behaviour. I wouldn't even call it a hack, it's trying to make Go's obsession with your filesystem actually usable -- why on earth does my compiler toolchain care about how my home directory is structured?
The old method still works, you just have to call the directory something other than vendor/. Which just defeats the whole point.
> Did you bring it up during the development of 1.5?
My experience with Go 1.5 was a lot of other (more serious) bugs, so I didn't have a chance to comment on that particular issue. I'm surprised nobody from Docker said anything though.
> I'm hoping that the packaging discussions that are going to be happening over the next few months don't result in a similar decision that "we know best".
Judging by current packaging discussions, I don't see it happening any other way. I just hope they don't mess things up too much.
- Open source, 6 years old, backing from Google
- Nice for concurrency and service implementations
- Not nice for generic types
- C like, modern, minimalist, garbage collected
- Becoming more popular
I wish the Clojure compiler was half as fast as Go's :(
I'm not much of a fan of using Go for anything 'big', but I have taken to using it in places where I would have previously used Python (tiny/simple services, housekeeping/utility scripts fired by cron, etc.)
I'd love to use Go at work (amongst many other things), but my employer already gives me a hard time for writing small utilities in Python rather than Java (I refuse to wait for a JVM to spin up just to convert a single file from csv to xml. I simply will not do it, and apparently my employer doesn't consider it worth firing me over).
Spinup of the JVM is 50 msec or so, for a hello world. Perhaps it's been a while since you tried running a small program on it? I can see why you might think the JVM starts slowly if you only ever run huge programs on it.
What makes you choose Go over Python for simple services and utility scripts? (I'm just learning Go but would heavily lean towards Python for those things)
I am looking forward to recompiling my code to see how things are. I have had my side project running on a set of Go micro services compiled against 1.4 since last year.
One benefit is that they were able to bootstrap the language and now enjoy having a pure Go toolchain.
Whereas with LLVM, while they could benefit from all the work that has gone into it, they would get a dependency on C++ for life.
This way, the progress is slower and multiple releases will be required to achieve some optimizations, on the other hand it is the only way to prove to naysayers that for the systems programming use case of writing compilers, Go is also a viable option.
It's not based on LLVM. I don't know how recently the official FAQ was updated, but they say [1] that while they thought about using LLVM, they thought it was too large and slow for what they were looking for.
There are llvm (and gcc) implementations of go but main/reference implementation was written from scratch in C and later moved to pure go. IIRC the authors wanted to achieve really fast compilation and a few other things that required them to start from scratch.
There was also the matter of figuring out llvm in order to use it. The initial Go compiler was based on the Plan 9 C compiler toolchain, which the designers of Go know well.
One of the other reasons relates to how stacks are handled for goroutines.
It's also a necessary "evil" to demonstrate that languages should have the right amount of features, no more and tools are part of a language, along with ease of deployment. It's is questionable whether Go strikes the right balance when it comes to features vs "simplicity" though, but it might inspire better solutions in the future. I think MSFT is trying to follow the same path with .net core and its tool chain, I hope they succeed as C# and F# are vastly superior to Go.
>I hope they succeed as C# and F# are vastly superior to Go.
In number of features, yes. Everything else and you come off as opinionated. I cannot comfortably write C# from my Linux workstation since all the tools are Windows-first.
C# is a language where everything must be done in classes and inherits a C++/Java school of OOP which I find abhorrent as it tends to favor monstrosities of abstraction upon abstraction. I can almost always read a Go codebase and get a good general sense of what it's doing in a few minutes, I can't say the same about C# codebases, and I have worked with C# for around the same time I've worked with Go.
These things aren't clear cut for everyone the same way.
I wrote C# professionally for 7 years. IMO, it's in almost way inferior to Go:
* requires a separate and quite heavy run-time
* still too windows-centric
* Enforces a class-based OOP style
* Functional programming elements feel very tacked-on
* Feels heavier, slower, more bureaucratic
* The async/await Task-based concurrency features are still not as flexible as Go's concurrency.
I find Go much more light-weight, easer to get into, enjoyable, and fun. So, how exactly is C# "vastly superior" to Go?
The semantics of Go are fine for the most part. My only problem with Go is manipulating slices. The rest is just awesome and it requires a lot of taste to simplify semantics without taking away the languages power.
Standout points in my opinion:
Overall performance improvements:
>> "We observed a 5–35% speedup across our benchmarks."
Decreased compile times and binary size:
>> "While these changes across the compiler toolchain are mostly invisible, users have observed a significant speedup in compile time and a reduction in binary size by as much as 20–30%."
Vendoring dependencies by default:
>> "...and in Go 1.7... the "vendor" behavior is always enabled"
Context package added to std lib:
>> "To make use of contexts within the standard library and to encourage more extensive use, the package has been moved from the x/net repository to the standard library as the context package."