Lorenzo Villani | 18 Sep 19:14 2014

TLS termination reverse proxy and performance issues

Hi there,

We are trying to build a TLS termination reverse proxy and load balancer in Go, while simultaneously
benchmarking it against Nginx 1.6.1.

Our first test was to configure both Nginx and our reverse proxy to be a plain simple HTTP reverse
proxy (no load balancing yet), with the Go version being a 3-lines source file which used
httputil.NewSingleHostReverseProxy(). Both Nginx and the Go version performed equally the same, as
expected.

Then, we tried to add SSL/TLS to the mix and we observed a significant drop in performance. We
configured both proxies to accept connections only over TLS 1.0, with different cipher suites
including RC4-SHA, using sslyze [1] to ensure that both servers were configured the same way.

Nginx always ran with a single worker process while we changed GOMAXPROCS between 1 and 8 and
whatever the default value is.

The test server has a 24-core (real + hyper-threading) Intel Xeon CPU with plenty of RAM. The test
"backend server" that sits behind Nginx and the Go proxy is a 5-line application which simply
replies with '42'.

We ran test with both 'ab' and blitz.io which consistently reported between 1.8x and ~3.0x
performance drop of the Go version compared to Nginx. We think this result is probably due to Go's
TLS stack, since with plain HTTP both Nginx and Go performed nearly the same, but we'd like to have
confirmation from the Go development team.

We tried both Go 1.3 and the current development version from Hg (revision b18ebcb9f236) with little
to no difference (the development tip gives slightly worse, and insignificant, results compared to
1.3).

Any idea on how we could improve the situation?

Thanks in advance


[1]: https://github.com/iSECPartners/sslyze

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Ondřej Kupka | 18 Sep 19:09 2014
Picon

Getting strange panic on method call

Hi,

I am getting a weird panic.

I have a variable called config, which is of type *Config.
Now I do config.SomeMethod() and I get panic being triggered by this line, the method is not entered.
However, I put a print there and config != nil. So how is it possible to get a panic in this case?
I get a panic even when I specify var config Config; config.SomeMethod()...

Will keep digging, looks too weird not to be my stupidity, but still asking...

Cheers,
Ondrej

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
michael.tiller | 18 Sep 03:58 2014
Picon

Comments about import paths and private repositories

I've been looking at some tooling for deployment and one thing I'd really like to have is the ability to use private repositories.  The issue that you quickly run into is that the "standard" for fetching Go dependencies is either "go get" or "godep" (which, as far as I can tell, uses "go get" or at least the "vcs" stuff beneath the surface).

So the bottom line is that I spent some time trying to figure out how to be able to use private repositories.  I found a lot of people with this issue, but many claim it can't be done or even that the Go documentation is incorrect.

Looking at the machinery, it seems that Go is perfectly capable of supporting private repositories but there are some really minor hang ups.

The first thing is that people wanting to use private repositories are generally expecting to use SSH.  As far as I can tell, the main hangup here is actually just the ordering of schemes in vcs.go.  The "https" scheme comes before the "git+ssh" scheme.  As a result, when it gets a hit on "git ls-remote https://..." it seems to think that is the preferred scheme.  If the authentication fails, it simply seems to give up.  The sad thing is that git+ssh is in the list (and this would be the best choice, as far as I can tell, for private repos) but it just never gets there (at least if you are using GitHub, since GitHub offers an https transport layer).

Was there a specific reason that https is "preferred" over git+ssh?  To be clear, GitHub supports the git+ssh scheme which means the current "go get" infrastructure would probably work just fine for dealing with private repositories (assuming you had the right SSH private key and perhaps a bit of setup in ~/.ssh/config).

On a somewhat related note, it is a bit of a shame that the import path stuff is so...rigid.  It doesn't seem to support a general scheme for turning import paths into URLs.  The documentation seems to imply that this:

import "<something>/repo.git"

is transformed into a url of the form:

<scheme>://<something>/repo

or

<scheme>://<something>/repo

where several schemes are tried.  However, there is clearly some logic behind the scenes because this doesn't work for:

something=identifier (i.e. a non-dot qualified name, despite the presence of .git explicitly in the import path)
something=hostname:port/path (i.e. if a port specification or even user specification is present in <something>)

I know there is special code to look for patterns related to providers like github and bitbucket.  But it seems like this interferes with other potentially legal URL components that are not from those providers.

Overall, I think that being able to easily utilize code in private repositories is important (at least it is important to me).  At the end of the day, it seems very useful to me that it should be possible to import (i.e. via "go get") code from private repositories.  Am I missing something?  Is anybody working on this?  Is there some reason (other than a lack of pull requests) why this wasn't done?

Thanks.

--
Mike

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
you fu | 18 Sep 16:35 2014
Picon

how to go run go code from github

I  go get github.com/skoo87/log4go
and then info 
➜  examples git:(master) pwd
/Users/fuyou/go/src/github.com/skoo87/log4go/examples
➜  examples git:(master) ll
total 0
drwxr-xr-x  4 fuyou  staff   136B  9 18 21:51 conf
drwxr-xr-x  3 fuyou  staff   102B  9 18 21:51 console
drwxr-xr-x  3 fuyou  staff   102B  9 18 21:51 file
drwxr-xr-x  3 fuyou  staff   102B  9 18 21:51 logger
drwxr-xr-x  3 fuyou  staff   102B  9 18 21:51 multi
drwxr-xr-x  3 fuyou  staff   102B  9 18 21:51 syslog
➜  examples git:(master) go build file/main.go
file/main.go:4:2: cannot find package "log4go" in any of:
/Users/fuyou/go/go/src/pkg/log4go (from $GOROOT)
/Users/fuyou/go/src/log4go (from $GOPATH)
➜  examples git:(master) echo $GOPATH
/Users/fuyou/go/
➜  examples git:(master) cd ..
➜  log4go git:(master) ls
LICENSE           README.md         config.go         console_writer.go examples          file_writer.go    log.go            syslog_writer.go
➜  log4go git:(master) cd ..
➜  skoo87  ls
log4go
➜  skoo87  cd ..
➜  github.com  ls
jstemmer nsf      skoo87
➜  github.com  cd ..
➜  src  ls
code.google.com github.com
➜  src  cd ..
➜  go  ls
bin    go     pkg    rog-go src
➜  go  go build github.com
can't load package: package github.com: no buildable Go source files in /Users/fuyou/go/src/github.com
➜  go  go build github.com/skoo87
can't load package: package github.com/skoo87: no buildable Go source files in /Users/fuyou/go/src/github.com/skoo87
➜  go  pwd
/Users/fuyou/go
➜  go  echo $GOPATH
/Users/fuyou/go/
➜  go  echo $GOROOT
/Users/fuyou/go/go
➜  go

and in another directory 

➜  examples git:(master) go run file/main.go
file/main.go:4:2: cannot find package "log4go" in any of:
/Users/fuyou/go/go/src/pkg/log4go (from $GOROOT)
/Users/fuyou/go/src/log4go (from $GOPATH)
➜  examples git:(master) go build
can't load package: package .: no buildable Go source files in /Users/fuyou/study/go/log4go/examples
➜  examples git:(master) pwd
/Users/fuyou/study/go/log4go/examples
➜  examples git:(master)


I read the article Go Build,but not instructions how to do from github. how to do ?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
omarshariffdontlikeit | 18 Sep 15:34 2014
Picon

scaling list of channels

I've implemented a scalable list of workers. The idea is that I want to be able to scale up and down (depending on the load across the rest of a system as a whole) the number of goroutine workers. The current version I have uses a slice of channels. Each channel is a kill channel, which is passed to the worker go routine. With that I can gracefully terminate the unneeded workers when scaling down, as well as add additional workers when more are needed.

I do this by creating a slice of kill channels, firing off the worker goroutines, then bulk append the kill channel slice to my master slice. When I want to scale down, I simply cut a slice off the end of the master slice, then iterate over the cut slice to send the kill signal to my workers.

https://gist.github.com/JalfResi/4948e082341277e5840b

My question is this: is this a good design? Is there anything I've missed? Could this be simpler? Any other observations?

Cheers!
Ben

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Paul van Brouwershaven | 18 Sep 14:33 2014

CreateCRL

The current version of x509.CreateCRL has no support for []pkix.Extension, which are required for adding the CRL Number and the Authority Key Id.

Now can I create a patch that would add the Authority Key Id by default as:

// Authority Key Id
var aki pkix.Extension
aki.Id = oidExtensionAuthorityKeyId
aki.Value, err = asn1.Marshal(authKeyId{Id: c.SubjectKeyId})
if err != nil {
return
}
tbsCertList.Extensions = append(tbsCertList.Extensions, aki)

But the CRL Number need to be specified by the user. And we also have some other extensions:

      5.2. CRL Extensions
           5.2.1. Authority Key Identifier
           5.2.2. Issuer Alternative Name
           5.2.3. CRL Number
           5.2.4. Delta CRL Indicator
           5.2.5. Issuing Distribution Point
           5.2.6. Freshest CRL (a.k.a. Delta CRL Distribution Point)
           5.2.7. Authority Information Access

I could add an []pkix.Extension to the function parameters but this would break some backwards compatibility.

Any suggestions how to implement this?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Dirk Struve | 18 Sep 13:49 2014
Picon

New project: Phylofriend for creating phylogenetic trees.

Phylofriend's main purpose is to calculate genetic distances using Y-STR (Short Tandem Repeats) values from the human Y chromosome. It makes it easy to extract data from Family Tree DNA projects and create phylogenetic trees showing the genetic relationships between people. It is a useful tool for anybody who is interested in genetic genealogy or anthropological research.

Phylofriend is located at https://code.google.com/p/phylofriend/.

Dirk

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Sankar | 18 Sep 13:42 2014
Picon

go-json-rest: Getting the username inside a REST handler while using AuthBasicMiddleware

Hi,

This query is only for the https://github.com/ant0ine/go-json-rest framework users. I am asking here in a hope that someone would have used the framework here.

I have the following go code:

handler := rest.ResourceHandler{
        PreRoutingMiddlewares: []rest.Middleware{
            &rest.AuthBasicMiddleware{
                Realm: "My Realm",
                Authenticator: func(userId string, password string) bool {
                    if userId == password {
                        /* Authenticate against openldap */
                        return true
                    }
                    return false
                },
            },
        },
    }

Now I am able to authenticate the incoming requests via my Authenticator, but I am not able to pass on this username information to any of the REST handlers. Is there a way to achieve this with the AuthBasicMiddleware at all ?

Thanks.

Sankar

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
Igor Avdoshkin | 18 Sep 10:56 2014
Picon

First experience in concurrency!

I try to write the first program but something is not working, what's wrong doing? Poke your finger ...

Code:
http://play.golang.org/p/Z1cBRKmSeu

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
chai2010 | 18 Sep 09:49 2014
Picon

confused with the interface's private method error message

This is the play code:

type TB int
func (t TB) Error(args ...interface{})                 {}
...
func (t TB) Skipped() bool                             { return false }
func (t TB) private() {}
var tb testing.TB = new(TB)

Error message:

prog.go:28: cannot use new(TB) (type *TB) as type testing.TB in assignment:
*TB does not implement testing.TB (missing testing.private method)
have private()
want testing.private()
[process exited with non-zero status]

But I think the `TB` implement testing.TB.

Thie `testing.TB` with `.private()` is not a really private interface.

I we need a really private interface, we can define the inteface like this:

type privateType int

type TB interface {
private(privateType)
}

The `privateType` is a really private type, so the outside user can't define
a `private(privateType)` method for the private `testing.TB` interface.

Thanks.


--

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.
John-Alan Simmons | 18 Sep 06:39 2014
Picon

Status of CGO?

Hello,

I am looking to get an overall status of CGO as it stands right now, with respect to Go 1.3. Its overall performance, overhead, support, future support, does it support C++, static binaries with C libs, gc vs gccgo. There are alot of really amazing libraries I want to incorporate into a project I am working on, which are large battle tested c libs that I don't necessarily want to port over to go right now, unless I have to.

Also I am not sure where, but I thought I heard that cgo calls, similar to native OS exec calls lock up a thread. Is this true?

I have googled around for some of these answers but I haven't found stuff written recently, and the cgo page on golang.org doesn't seem to have been updated, or maybe things just haven't changed.

Thanks

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit https://groups.google.com/d/optout.

Gmane