add 1.4.3 changelog (#4321)

- Fix typo that caused Gogs hooks to be created as Gitea hooks.
- Fix typo that caused Gogs hooks to be duplicated upon edit (though this bug was masked by the previous one).

Signed-off-by: Russell Aunger <rba@live.com>

CHANGELOG.md

@@ -4,11 +4,59 @@ This changelog goes through all the changes that have been made in each release
 without substantial changes to our git log; to see the highlights of what has
 been added to each release, please refer to the [blog](https://blog.gitea.io).
 
-## [1.4.0-rc1](https://github.com/go-gitea/gitea/releases/tag/v1.4.0-rc1) - 2018-01-31
+## [1.4.3](https://github.com/go-gitea/gitea/releases/tag/v1.4.3) - 2018-06-26
+* SECURITY
+  * HTML-escape plain-text READMEs (#4192) (#4214)
+  * Fix open redirect vulnerability on login screen (#4312) (#4312)
+* BUGFIXES
+  * Fix broken monitoring page when running processes are shown (#4203) (#4208)
+  * Fix delete comment bug (#4216) (#4228)
+  * Delete reactions added to issues and comments when deleting repository (#4232) (#4237)
+  * Fix wiki URL encoding bug (#4091) (#4254)
+  * Fix code tab link when viewing tags (#3908) (#4263)
+  * Fix webhook type conflation (#4285) (#4285)
+
+## [1.4.2](https://github.com/go-gitea/gitea/releases/tag/v1.4.2) - 2018-06-04
+* BUGFIXES
+  * Adjust z-index for floating labels (#3939) (#3950)
+  * Add missing token validation on application settings page (#3976) #3978
+  * Webhook and hook_task clean up (#4006)
+  * Fix webhook bug of response info is not displayed in UI (#4023)
+  * Fix writer cannot read bare repo guide (#4033) (#4039)
+  * Don't force due date to current time (#3830) (#4057)
+  * Fix wiki redirects (#3919) (#4065)
+  * Fix attachment ENABLED (#4064) (#4066)
+  * Added deletion of an empty line at the end of file (#4054) (#4074)
+  * Use ResolveReference instead of path.Join (#4073)
+  * Fix #4081 Check for leading / in base before removing it (#4083)
+  * Respository's home page not updated after first push (#4075)
+
+## [1.4.1](https://github.com/go-gitea/gitea/releases/tag/v1.4.1) - 2018-05-03
+* BREAKING
+  * Add "error" as reserved username (#3882) (#3886)
+* SECURITY
+  * Do not allow inactive users to access repositories using private key (#3887) (#3889)
+  * Fix path cleanup in file editor, when initilizing new repository and LFS oids  (#3871) (#3873)
+  * Remove unnecessary allowed safe HTML (#3778) (#3779)
+  * Correctly check http git access rights for reverse proxy authorized users (#3721) (#3743)
+* BUGFIXES
+  * Fix to use only needed columns from tables to get repository git paths (#3870) (#3883)
+  * Fix GPG expire time display when time is zero (#3584) (#3884)
+  * Fix to update only issue last update time when adding a comment (#3855) (#3860)
+  * Fix repository star count after deleting user (#3781) (#3783)
+  * Use the active branch for the code tab (#3720) (#3776)
+  * Set default branch name on first push (#3715) (#3723)
+  * Show clipboard button if disable HTTP of git protocol (#3773) (#3774)
+
+## [1.4.0](https://github.com/go-gitea/gitea/releases/tag/v1.4.0) - 2018-03-25
 * BREAKING
   * Drop deprecated GOGS\_WORK\_DIR use (#2946)
   * Fix API status code for hook creation (#2814)
 * SECURITY
+  * Escape branch name in dropdown menu (#3691) (#3692)
+  * Refactor and simplify to correctly validate redirect to URL (#3674) (#3676)
+  * Fix escaping changed title in comments (#3530) (#3534)
+  * Escape search query (#3486) (#3488)
   * Sanitize logs for mirror sync (#3057)
 * FEATURE
   * Serve .patch and .diff for pull requests (#3305, #3293)
@@ -24,6 +72,17 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
   * Add dingtalk webhook  (#2777)
   * Responsive view (#2750)
 * BUGFIXES
+  * Fix wiki inter-links with spaces (#3560) (#3632)
+  * Fix query protected branch bug (#3563) (#3571)
+  * Fix remove team member issue (#3566) (#3570)
+  * Fix the protected branch panic issue (#3567) (#3569)
+  * If Mirrors repository no content is fetched, updated time should not be changed (#3551) (#3565)
+  * Bug fix for mirrored repository releases sorted (#3522) (#3555)
+  * Add issue closed time column to fix activity closed issues list (#3537) (#3540)
+  * Update markbates/goth library to support OAuth2 with new dropbox API (#3533) (#3539)
+  * Fixes missing avatars in offline mode (#3471) (#3477)
+  * Fix synchronization bug in repo indexer (#3455) (#3461)
+  * Fix rendering of wiki page list if wiki repo contains other files (#3454) (#3463)
   * Fix webhook X-GitHub-* headers casing for better compatibility (#3429)
   * Add content type and doctype to requests made with go-get (#3426, #3423)
   * Fix SQL type error for webhooks (#3424)

cmd/serv.go

@@ -230,6 +230,12 @@ func runServ(c *cli.Context) error {
 				fail("internal error", "Failed to get user by key ID(%d): %v", keyID, err)
 			}
 
+			if !user.IsActive || user.ProhibitLogin {
+				fail("Your account is not active or has been disabled by Administrator",
+					"User %s is disabled and have no access to repository %s",
+					user.Name, repoPath)
+			}
+
 			mode, err := models.AccessLevel(user.ID, repo)
 			if err != nil {
 				fail("Internal error", "Failed to check access: %v", err)

custom/conf/app.ini.sample

@@ -403,7 +403,7 @@ ENABLE_FEDERATED_AVATAR = false
 
 [attachment]
 ; Whether attachments are enabled. Defaults to `true`
-ENABLE = true
+ENABLED = true
 ; Path for attachments. Defaults to `data/attachments`
 PATH = data/attachments
 ; One or more allowed types, e.g. image/jpeg|image/png

docs/content/doc/advanced/config-cheat-sheet.en-us.md

@@ -128,7 +128,7 @@ Values containing `#` or `;` must be quoted using `` ` `` or `"""`.
 - `REPO_INDEXER_ENABLED`: **false**: Enables code search (uses a lot of disk space).
 - `REPO_INDEXER_PATH`: **indexers/repos.bleve**: Index file used for code search.
 - `UPDATE_BUFFER_LEN`: **20**: Buffer length of index request.
-- `MAX_FILE_SIZE`: **1048576**: Maximum size in bytes of each index files.
+- `MAX_FILE_SIZE`: **1048576**: Maximum size in bytes of files to be indexed.
 
 ## Security (`security`)
 

docs/content/page/index.en-us.md

@@ -2,6 +2,7 @@
 date: "2016-11-08T16:00:00+02:00"
 title: "Documentation"
 slug: "documentation"
+url: "/en-us/"
 weight: 10
 toc: true
 draft: false

docs/content/page/index.fr-fr.md

@@ -2,6 +2,7 @@
 date: "2017-08-23T09:00:00+02:00"
 title: "Documentation"
 slug: "documentation"
+url: "/fr-fr/"
 weight: 10
 toc: true
 draft: false
@@ -48,7 +49,7 @@ Le but de ce projet est de fournir de la manière la plus simple, la plus rapide
     - Migré
 - Notifications (courriel et web)
     - Lu
-    - Non lu 
+    - Non lu
     - Épinglé
 - Page d'exploration
     - Utilisateurs

docs/content/page/index.zh-cn.md

@@ -2,6 +2,7 @@
 date: "2016-11-08T16:00:00+02:00"
 title: "文档"
 slug: "documentation"
+url: "/zh-cn/"
 weight: 10
 toc: true
 draft: false

docs/content/page/index.zh-tw.md

@@ -2,6 +2,7 @@
 date: "2016-11-08T16:00:00+02:00"
 title: "文件"
 slug: "documentation"
+url: "/zh-tw/"
 weight: 10
 toc: true
 draft: false

docs/scripts/trans-copy

@@ -28,6 +28,8 @@ for SOURCE in $(find ${ROOT}/content -type f -iname *.en-us.md); do
     if [[ ! -f ${DEST} ]]; then
       echo "Creating fallback for ${DEST#${ROOT}/content/}"
       cp ${SOURCE} ${DEST}
+      sed -i.bak "s/en\-us/${LOCALE}/g" ${DEST}
+      rm ${DEST}.bak
     fi
   done
 done

integrations/pull_create_test.go

@@ -14,7 +14,7 @@ import (
 	"github.com/stretchr/testify/assert"
 )
 
-func testPullCreate(t *testing.T, session *TestSession, user, repo, branch string) *httptest.ResponseRecorder {
+func testPullCreate(t *testing.T, session *TestSession, user, repo, branch, title string) *httptest.ResponseRecorder {
 	req := NewRequest(t, "GET", path.Join(user, repo))
 	resp := session.MakeRequest(t, req, http.StatusOK)
 
@@ -35,7 +35,7 @@ func testPullCreate(t *testing.T, session *TestSession, user, repo, branch strin
 	assert.True(t, exists, "The template has changed")
 	req = NewRequestWithValues(t, "POST", link, map[string]string{
 		"_csrf": htmlDoc.GetCSRF(),
-		"title": "This is a pull title",
+		"title": title,
 	})
 	resp = session.MakeRequest(t, req, http.StatusFound)
 
@@ -47,7 +47,7 @@ func TestPullCreate(t *testing.T) {
 	session := loginUser(t, "user1")
 	testRepoFork(t, session, "user2", "repo1", "user1", "repo1")
 	testEditFile(t, session, "user1", "repo1", "master", "README.md", "Hello, World (Edited)\n")
-	resp := testPullCreate(t, session, "user1", "repo1", "master")
+	resp := testPullCreate(t, session, "user1", "repo1", "master", "This is a pull title")
 
 	// check the redirected URL
 	url := resp.HeaderMap.Get("Location")
@@ -68,3 +68,38 @@ func TestPullCreate(t *testing.T) {
 	assert.Regexp(t, `Subject: \[PATCH\] Update 'README.md'`, resp.Body)
 	assert.NotRegexp(t, "diff.*diff", resp.Body) // not two diffs, just one
 }
+
+func TestPullCreate_TitleEscape(t *testing.T) {
+	prepareTestEnv(t)
+	session := loginUser(t, "user1")
+	testRepoFork(t, session, "user2", "repo1", "user1", "repo1")
+	testEditFile(t, session, "user1", "repo1", "master", "README.md", "Hello, World (Edited)\n")
+	resp := testPullCreate(t, session, "user1", "repo1", "master", "<i>XSS PR</i>")
+
+	// check the redirected URL
+	url := resp.HeaderMap.Get("Location")
+	assert.Regexp(t, "^/user2/repo1/pulls/[0-9]*$", url)
+
+	// Edit title
+	req := NewRequest(t, "GET", url)
+	resp = session.MakeRequest(t, req, http.StatusOK)
+	htmlDoc := NewHTMLParser(t, resp.Body)
+	editTestTitleURL, exists := htmlDoc.doc.Find("#save-edit-title").First().Attr("data-update-url")
+	assert.True(t, exists, "The template has changed")
+
+	req = NewRequestWithValues(t, "POST", editTestTitleURL, map[string]string{
+		"_csrf": htmlDoc.GetCSRF(),
+		"title": "<u>XSS PR</u>",
+	})
+	session.MakeRequest(t, req, http.StatusOK)
+
+	req = NewRequest(t, "GET", url)
+	resp = session.MakeRequest(t, req, http.StatusOK)
+	htmlDoc = NewHTMLParser(t, resp.Body)
+	titleHTML, err := htmlDoc.doc.Find(".comments .event .text b").First().Html()
+	assert.NoError(t, err)
+	assert.Equal(t, "&lt;i&gt;XSS PR&lt;/i&gt;", titleHTML)
+	titleHTML, err = htmlDoc.doc.Find(".comments .event .text b").Next().Html()
+	assert.NoError(t, err)
+	assert.Equal(t, "&lt;u&gt;XSS PR&lt;/u&gt;", titleHTML)
+}

integrations/pull_merge_test.go

@@ -56,7 +56,7 @@ func TestPullMerge(t *testing.T) {
 	testRepoFork(t, session, "user2", "repo1", "user1", "repo1")
 	testEditFile(t, session, "user1", "repo1", "master", "README.md", "Hello, World (Edited)\n")
 
-	resp := testPullCreate(t, session, "user1", "repo1", "master")
+	resp := testPullCreate(t, session, "user1", "repo1", "master", "This is a pull title")
 
 	elem := strings.Split(test.RedirectURL(resp), "/")
 	assert.EqualValues(t, "pulls", elem[3])
@@ -69,7 +69,7 @@ func TestPullRebase(t *testing.T) {
 	testRepoFork(t, session, "user2", "repo1", "user1", "repo1")
 	testEditFile(t, session, "user1", "repo1", "master", "README.md", "Hello, World (Edited)\n")
 
-	resp := testPullCreate(t, session, "user1", "repo1", "master")
+	resp := testPullCreate(t, session, "user1", "repo1", "master", "This is a pull title")
 
 	elem := strings.Split(test.RedirectURL(resp), "/")
 	assert.EqualValues(t, "pulls", elem[3])
@@ -83,7 +83,7 @@ func TestPullSquash(t *testing.T) {
 	testEditFile(t, session, "user1", "repo1", "master", "README.md", "Hello, World (Edited)\n")
 	testEditFile(t, session, "user1", "repo1", "master", "README.md", "Hello, World (Edited!)\n")
 
-	resp := testPullCreate(t, session, "user1", "repo1", "master")
+	resp := testPullCreate(t, session, "user1", "repo1", "master", "This is a pull title")
 
 	elem := strings.Split(test.RedirectURL(resp), "/")
 	assert.EqualValues(t, "pulls", elem[3])
@@ -96,7 +96,7 @@ func TestPullCleanUpAfterMerge(t *testing.T) {
 	testRepoFork(t, session, "user2", "repo1", "user1", "repo1")
 	testEditFileToNewBranch(t, session, "user1", "repo1", "master", "feature/test", "README.md", "Hello, World (Edited)\n")
 
-	resp := testPullCreate(t, session, "user1", "repo1", "feature/test")
+	resp := testPullCreate(t, session, "user1", "repo1", "feature/test", "This is a pull title")
 
 	elem := strings.Split(test.RedirectURL(resp), "/")
 	assert.EqualValues(t, "pulls", elem[3])

integrations/repo_activity_test.go

@@ -22,16 +22,16 @@ func TestRepoActivity(t *testing.T) {
 	// Create PRs (1 merged & 2 proposed)
 	testRepoFork(t, session, "user2", "repo1", "user1", "repo1")
 	testEditFile(t, session, "user1", "repo1", "master", "README.md", "Hello, World (Edited)\n")
-	resp := testPullCreate(t, session, "user1", "repo1", "master")
+	resp := testPullCreate(t, session, "user1", "repo1", "master", "This is a pull title")
 	elem := strings.Split(test.RedirectURL(resp), "/")
 	assert.EqualValues(t, "pulls", elem[3])
 	testPullMerge(t, session, elem[1], elem[2], elem[4], models.MergeStyleMerge)
 
 	testEditFileToNewBranch(t, session, "user1", "repo1", "master", "feat/better_readme", "README.md", "Hello, World (Edited Again)\n")
-	testPullCreate(t, session, "user1", "repo1", "feat/better_readme")
+	testPullCreate(t, session, "user1", "repo1", "feat/better_readme", "This is a pull title")
 
 	testEditFileToNewBranch(t, session, "user1", "repo1", "master", "feat/much_better_readme", "README.md", "Hello, World (Edited More)\n")
-	testPullCreate(t, session, "user1", "repo1", "feat/much_better_readme")
+	testPullCreate(t, session, "user1", "repo1", "feat/much_better_readme", "This is a pull title")
 
 	// Create issues (3 new issues)
 	testNewIssue(t, session, "user2", "repo1", "Issue 1", "Description 1")

models/action.go

@@ -523,6 +523,11 @@ func CommitRepoAction(opts CommitRepoActionOptions) error {
 		return fmt.Errorf("GetRepositoryByName [owner_id: %d, name: %s]: %v", opts.RepoOwnerID, opts.RepoName, err)
 	}
 
+	refName := git.RefEndName(opts.RefFullName)
+	if repo.IsBare && refName != repo.DefaultBranch {
+		repo.DefaultBranch = refName
+	}
+
 	// Change repository bare status and update last updated time.
 	repo.IsBare = repo.IsBare && opts.Commits.Len <= 0
 	if err = UpdateRepository(repo, false); err != nil {
@@ -563,7 +568,6 @@ func CommitRepoAction(opts CommitRepoActionOptions) error {
 		return fmt.Errorf("Marshal: %v", err)
 	}
 
-	refName := git.RefEndName(opts.RefFullName)
 	if err = NotifyWatchers(&Action{
 		ActUserID: pusher.ID,
 		ActUser:   pusher,
@@ -742,5 +746,14 @@ func GetFeeds(opts GetFeedsOptions) ([]*Action, error) {
 	}
 
 	actions := make([]*Action, 0, 20)
-	return actions, x.Limit(20).Desc("id").Where(cond).Find(&actions)
+
+	if err := x.Limit(20).Desc("id").Where(cond).Find(&actions); err != nil {
+		return nil, fmt.Errorf("Find: %v", err)
+	}
+
+	if err := ActionList(actions).LoadAttributes(); err != nil {
+		return nil, fmt.Errorf("LoadAttributes: %v", err)
+	}
+
+	return actions, nil
 }

models/action_list.go

@@ -0,0 +1,98 @@
+// Copyright 2018 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package models
+
+import "fmt"
+
+// ActionList defines a list of actions
+type ActionList []*Action
+
+func (actions ActionList) getUserIDs() []int64 {
+	userIDs := make(map[int64]struct{}, len(actions))
+	for _, action := range actions {
+		if _, ok := userIDs[action.ActUserID]; !ok {
+			userIDs[action.ActUserID] = struct{}{}
+		}
+	}
+	return keysInt64(userIDs)
+}
+
+func (actions ActionList) loadUsers(e Engine) ([]*User, error) {
+	if len(actions) == 0 {
+		return nil, nil
+	}
+
+	userIDs := actions.getUserIDs()
+	userMaps := make(map[int64]*User, len(userIDs))
+	err := e.
+		In("id", userIDs).
+		Find(&userMaps)
+	if err != nil {
+		return nil, fmt.Errorf("find user: %v", err)
+	}
+
+	for _, action := range actions {
+		action.ActUser = userMaps[action.ActUserID]
+	}
+	return valuesUser(userMaps), nil
+}
+
+// LoadUsers loads actions' all users
+func (actions ActionList) LoadUsers() ([]*User, error) {
+	return actions.loadUsers(x)
+}
+
+func (actions ActionList) getRepoIDs() []int64 {
+	repoIDs := make(map[int64]struct{}, len(actions))
+	for _, action := range actions {
+		if _, ok := repoIDs[action.RepoID]; !ok {
+			repoIDs[action.RepoID] = struct{}{}
+		}
+	}
+	return keysInt64(repoIDs)
+}
+
+func (actions ActionList) loadRepositories(e Engine) ([]*Repository, error) {
+	if len(actions) == 0 {
+		return nil, nil
+	}
+
+	repoIDs := actions.getRepoIDs()
+	repoMaps := make(map[int64]*Repository, len(repoIDs))
+	err := e.
+		In("id", repoIDs).
+		Find(&repoMaps)
+	if err != nil {
+		return nil, fmt.Errorf("find repository: %v", err)
+	}
+
+	for _, action := range actions {
+		action.Repo = repoMaps[action.RepoID]
+	}
+	return valuesRepository(repoMaps), nil
+}
+
+// LoadRepositories loads actions' all repositories
+func (actions ActionList) LoadRepositories() ([]*Repository, error) {
+	return actions.loadRepositories(x)
+}
+
+// loadAttributes loads all attributes
+func (actions ActionList) loadAttributes(e Engine) (err error) {
+	if _, err = actions.loadUsers(e); err != nil {
+		return
+	}
+
+	if _, err = actions.loadRepositories(e); err != nil {
+		return
+	}
+
+	return nil
+}
+
+// LoadAttributes loads attributes of the actions
+func (actions ActionList) LoadAttributes() error {
+	return actions.loadAttributes(x)
+}

models/branches.go

@@ -6,7 +6,6 @@ package models
 
 import (
 	"fmt"
-	"strings"
 	"time"
 
 	"code.gitea.io/gitea/modules/base"
@@ -70,7 +69,7 @@ func GetProtectedBranchByRepoID(RepoID int64) ([]*ProtectedBranch, error) {
 
 // GetProtectedBranchBy getting protected branch by ID/Name
 func GetProtectedBranchBy(repoID int64, BranchName string) (*ProtectedBranch, error) {
-	rel := &ProtectedBranch{RepoID: repoID, BranchName: strings.ToLower(BranchName)}
+	rel := &ProtectedBranch{RepoID: repoID, BranchName: BranchName}
 	has, err := x.Get(rel)
 	if err != nil {
 		return nil, err
@@ -156,6 +155,10 @@ func (repo *Repository) GetProtectedBranches() ([]*ProtectedBranch, error) {
 
 // IsProtectedBranch checks if branch is protected
 func (repo *Repository) IsProtectedBranch(branchName string, doer *User) (bool, error) {
+	if doer == nil {
+		return true, nil
+	}
+
 	protectedBranch := &ProtectedBranch{
 		RepoID:     repo.ID,
 		BranchName: branchName,

models/error.go

@@ -216,6 +216,21 @@ func (err ErrWikiReservedName) Error() string {
 	return fmt.Sprintf("wiki title is reserved: %s", err.Title)
 }
 
+// ErrWikiInvalidFileName represents an invalid wiki file name.
+type ErrWikiInvalidFileName struct {
+	FileName string
+}
+
+// IsErrWikiInvalidFileName checks if an error is an ErrWikiInvalidFileName.
+func IsErrWikiInvalidFileName(err error) bool {
+	_, ok := err.(ErrWikiInvalidFileName)
+	return ok
+}
+
+func (err ErrWikiInvalidFileName) Error() string {
+	return fmt.Sprintf("Invalid wiki filename: %s", err.FileName)
+}
+
 // __________     ___.   .__  .__          ____  __.
 // \______   \__ _\_ |__ |  | |__| ____   |    |/ _|____ ___.__.
 //  |     ___/  |  \ __ \|  | |  |/ ___\  |      <_/ __ <   |  |

models/git_diff.go

@@ -14,6 +14,7 @@ import (
 	"io/ioutil"
 	"os"
 	"os/exec"
+	"strconv"
 	"strings"
 
 	"code.gitea.io/git"
@@ -368,8 +369,15 @@ func ParsePatch(maxLines, maxLineCharacters, maxFiles int, reader io.Reader) (*D
 			a := line[beg+2 : middle]
 			b := line[middle+3:]
 			if hasQuote {
-				a = string(git.UnescapeChars([]byte(a[1 : len(a)-1])))
-				b = string(git.UnescapeChars([]byte(b[1 : len(b)-1])))
+				var err error
+				a, err = strconv.Unquote(a)
+				if err != nil {
+					return nil, fmt.Errorf("Unquote: %v", err)
+				}
+				b, err = strconv.Unquote(b)
+				if err != nil {
+					return nil, fmt.Errorf("Unquote: %v", err)
+				}
 			}
 
 			curFile = &DiffFile{

models/git_diff_test.go

@@ -21,16 +21,16 @@ func assertLineEqual(t *testing.T, d1 *DiffLine, d2 *DiffLine) {
 
 func TestDiffToHTML(t *testing.T) {
 	assertEqual(t, "+foo <span class=\"added-code\">bar</span> biz", diffToHTML([]dmp.Diff{
-		{dmp.DiffEqual, "foo "},
-		{dmp.DiffInsert, "bar"},
-		{dmp.DiffDelete, " baz"},
-		{dmp.DiffEqual, " biz"},
+		{Type: dmp.DiffEqual, Text: "foo "},
+		{Type: dmp.DiffInsert, Text: "bar"},
+		{Type: dmp.DiffDelete, Text: " baz"},
+		{Type: dmp.DiffEqual, Text: " biz"},
 	}, DiffLineAdd))
 
 	assertEqual(t, "-foo <span class=\"removed-code\">bar</span> biz", diffToHTML([]dmp.Diff{
-		{dmp.DiffEqual, "foo "},
-		{dmp.DiffDelete, "bar"},
-		{dmp.DiffInsert, " baz"},
-		{dmp.DiffEqual, " biz"},
+		{Type: dmp.DiffEqual, Text: "foo "},
+		{Type: dmp.DiffDelete, Text: "bar"},
+		{Type: dmp.DiffInsert, Text: " baz"},
+		{Type: dmp.DiffEqual, Text: " biz"},
 	}, DiffLineDel))
 }

models/issue.go

@@ -49,6 +49,7 @@ type Issue struct {
 	DeadlineUnix util.TimeStamp `xorm:"INDEX"`
 	CreatedUnix  util.TimeStamp `xorm:"INDEX created"`
 	UpdatedUnix  util.TimeStamp `xorm:"INDEX updated"`
+	ClosedUnix   util.TimeStamp `xorm:"INDEX"`
 
 	Attachments []*Attachment `xorm:"-"`
 	Comments    []*Comment    `xorm:"-"`
@@ -612,8 +613,13 @@ func (issue *Issue) changeStatus(e *xorm.Session, doer *User, repo *Repository,
 		return nil
 	}
 	issue.IsClosed = isClosed
+	if isClosed {
+		issue.ClosedUnix = util.TimeStampNow()
+	} else {
+		issue.ClosedUnix = 0
+	}
 
-	if err = updateIssueCols(e, issue, "is_closed"); err != nil {
+	if err = updateIssueCols(e, issue, "is_closed", "closed_unix"); err != nil {
 		return err
 	}
 

models/issue_comment.go

@@ -132,6 +132,10 @@ func (c *Comment) AfterLoad(session *xorm.Session) {
 
 // AfterDelete is invoked from XORM after the object is deleted.
 func (c *Comment) AfterDelete() {
+	if c.ID <= 0 {
+		return
+	}
+
 	_, err := DeleteAttachmentsByComment(c.ID, true)
 
 	if err != nil {
@@ -418,7 +422,7 @@ func createComment(e *xorm.Session, opts *CreateCommentOptions) (_ *Comment, err
 	}
 
 	// update the issue's updated_unix column
-	if err = updateIssueCols(e, opts.Issue); err != nil {
+	if err = updateIssueCols(e, opts.Issue, "updated_unix"); err != nil {
 		return nil, err
 	}
 

models/issue_milestone.go

@@ -31,11 +31,6 @@ type Milestone struct {
 	ClosedDateUnix util.TimeStamp
 }
 
-// BeforeInsert is invoked from XORM before inserting an object of this type.
-func (m *Milestone) BeforeInsert() {
-	m.DeadlineUnix = util.TimeStampNow()
-}
-
 // BeforeUpdate is invoked from XORM before updating this object.
 func (m *Milestone) BeforeUpdate() {
 	if m.NumIssues > 0 {

models/issue_milestone_test.go

@@ -214,13 +214,15 @@ func TestChangeMilestoneIssueStats(t *testing.T) {
 		"is_closed=0").(*Issue)
 
 	issue.IsClosed = true
-	_, err := x.Cols("is_closed").Update(issue)
+	issue.ClosedUnix = util.TimeStampNow()
+	_, err := x.Cols("is_closed", "closed_unix").Update(issue)
 	assert.NoError(t, err)
 	assert.NoError(t, changeMilestoneIssueStats(x.NewSession(), issue))
 	CheckConsistencyFor(t, &Milestone{})
 
 	issue.IsClosed = false
-	_, err = x.Cols("is_closed").Update(issue)
+	issue.ClosedUnix = 0
+	_, err = x.Cols("is_closed", "closed_unix").Update(issue)
 	assert.NoError(t, err)
 	assert.NoError(t, changeMilestoneIssueStats(x.NewSession(), issue))
 	CheckConsistencyFor(t, &Milestone{})

models/migrations/migrations.go

@@ -166,6 +166,8 @@ var migrations = []Migration{
 	NewMigration("add writable deploy keys", addModeToDeploKeys),
 	// v56 -> v57
 	NewMigration("remove is_owner, num_teams columns from org_user", removeIsOwnerColumnFromOrgUser),
+	// v57 -> v58
+	NewMigration("add closed_unix column for issues", addIssueClosedTime),
 }
 
 // Migrate database to current version
@@ -215,6 +217,66 @@ Please try to upgrade to a lower version (>= v0.6.0) first, then upgrade to curr
 	return nil
 }
 
+func dropTableColumns(x *xorm.Engine, tableName string, columnNames ...string) (err error) {
+	if tableName == "" || len(columnNames) == 0 {
+		return nil
+	}
+
+	switch {
+	case setting.UseSQLite3:
+		log.Warn("Unable to drop columns in SQLite")
+	case setting.UseMySQL, setting.UseTiDB, setting.UsePostgreSQL:
+		cols := ""
+		for _, col := range columnNames {
+			if cols != "" {
+				cols += ", "
+			}
+			cols += "DROP COLUMN `" + col + "`"
+		}
+		if _, err := x.Exec(fmt.Sprintf("ALTER TABLE `%s` %s", tableName, cols)); err != nil {
+			return fmt.Errorf("Drop table `%s` columns %v: %v", tableName, columnNames, err)
+		}
+	case setting.UseMSSQL:
+		sess := x.NewSession()
+		defer sess.Close()
+
+		if err = sess.Begin(); err != nil {
+			return err
+		}
+
+		cols := ""
+		for _, col := range columnNames {
+			if cols != "" {
+				cols += ", "
+			}
+			cols += "`" + strings.ToLower(col) + "`"
+		}
+		sql := fmt.Sprintf("SELECT Name FROM SYS.DEFAULT_CONSTRAINTS WHERE PARENT_OBJECT_ID = OBJECT_ID('%[1]s') AND PARENT_COLUMN_ID IN (SELECT column_id FROM sys.columns WHERE lower(NAME) IN (%[2]s) AND object_id = OBJECT_ID('%[1]s'))",
+			tableName, strings.Replace(cols, "`", "'", -1))
+		constraints := make([]string, 0)
+		if err := sess.SQL(sql).Find(&constraints); err != nil {
+			sess.Rollback()
+			return fmt.Errorf("Find constraints: %v", err)
+		}
+		for _, constraint := range constraints {
+			if _, err := sess.Exec(fmt.Sprintf("ALTER TABLE `%s` DROP CONSTRAINT `%s`", tableName, constraint)); err != nil {
+				sess.Rollback()
+				return fmt.Errorf("Drop table `%s` constraint `%s`: %v", tableName, constraint, err)
+			}
+		}
+		if _, err := sess.Exec(fmt.Sprintf("ALTER TABLE `%s` DROP COLUMN %s", tableName, cols)); err != nil {
+			sess.Rollback()
+			return fmt.Errorf("Drop table `%s` columns %v: %v", tableName, columnNames, err)
+		}
+
+		return sess.Commit()
+	default:
+		log.Fatal(4, "Unrecognized DB")
+	}
+
+	return nil
+}
+
 func fixLocaleFileLoadPanic(_ *xorm.Engine) error {
 	cfg, err := ini.Load(setting.CustomConf)
 	if err != nil {

models/migrations/v56.go

@@ -5,29 +5,9 @@
 package migrations
 
 import (
-	"fmt"
-
-	"code.gitea.io/gitea/modules/log"
-	"code.gitea.io/gitea/modules/setting"
-
 	"github.com/go-xorm/xorm"
 )
 
 func removeIsOwnerColumnFromOrgUser(x *xorm.Engine) (err error) {
-	switch {
-	case setting.UseSQLite3:
-		log.Warn("Unable to drop columns in SQLite")
-	case setting.UseMySQL, setting.UseTiDB, setting.UsePostgreSQL:
-		if _, err := x.Exec("ALTER TABLE org_user DROP COLUMN is_owner, DROP COLUMN num_teams"); err != nil {
-			return fmt.Errorf("DROP COLUMN org_user.is_owner, org_user.num_teams: %v", err)
-		}
-	case setting.UseMSSQL:
-		if _, err := x.Exec("ALTER TABLE org_user DROP COLUMN is_owner, num_teams"); err != nil {
-			return fmt.Errorf("DROP COLUMN org_user.is_owner, org_user.num_teams: %v", err)
-		}
-	default:
-		log.Fatal(4, "Unrecognized DB")
-	}
-
-	return nil
+	return dropTableColumns(x, "org_user", "is_owner", "num_teams")
 }

models/migrations/v57.go

@@ -0,0 +1,30 @@
+// Copyright 2017 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package migrations
+
+import (
+	"fmt"
+
+	"code.gitea.io/gitea/modules/util"
+
+	"github.com/go-xorm/xorm"
+)
+
+func addIssueClosedTime(x *xorm.Engine) error {
+	// Issue see models/issue.go
+	type Issue struct {
+		ClosedUnix util.TimeStamp `xorm:"INDEX"`
+	}
+
+	if err := x.Sync2(new(Issue)); err != nil {
+		return fmt.Errorf("Sync2: %v", err)
+	}
+
+	if _, err := x.Exec("UPDATE `issue` SET `closed_unix` = `updated_unix` WHERE `is_closed` = ?", true); err != nil {
+		return err
+	}
+
+	return nil
+}

models/org.go

@@ -436,8 +436,7 @@ func AddOrgUser(orgID, uid int64) error {
 	return sess.Commit()
 }
 
-// RemoveOrgUser removes user from given organization.
-func RemoveOrgUser(orgID, userID int64) error {
+func removeOrgUser(sess *xorm.Session, orgID, userID int64) error {
 	ou := new(OrgUser)
 
 	has, err := x.
@@ -473,12 +472,6 @@ func RemoveOrgUser(orgID, userID int64) error {
 		}
 	}
 
-	sess := x.NewSession()
-	defer sess.Close()
-	if err := sess.Begin(); err != nil {
-		return err
-	}
-
 	if _, err := sess.ID(ou.ID).Delete(ou); err != nil {
 		return err
 	} else if _, err = sess.Exec("UPDATE `user` SET num_members=num_members-1 WHERE id=?", orgID); err != nil {
@@ -520,6 +513,19 @@ func RemoveOrgUser(orgID, userID int64) error {
 		}
 	}
 
+	return nil
+}
+
+// RemoveOrgUser removes user from given organization.
+func RemoveOrgUser(orgID, userID int64) error {
+	sess := x.NewSession()
+	defer sess.Close()
+	if err := sess.Begin(); err != nil {
+		return err
+	}
+	if err := removeOrgUser(sess, orgID, userID); err != nil {
+		return err
+	}
 	return sess.Commit()
 }
 

models/org_team.go

@@ -10,6 +10,8 @@ import (
 	"strings"
 
 	"code.gitea.io/gitea/modules/log"
+
+	"github.com/go-xorm/xorm"
 )
 
 const ownerTeamName = "Owners"
@@ -521,7 +523,7 @@ func AddTeamMember(team *Team, userID int64) error {
 	return sess.Commit()
 }
 
-func removeTeamMember(e Engine, team *Team, userID int64) error {
+func removeTeamMember(e *xorm.Session, team *Team, userID int64) error {
 	isMember, err := isTeamMember(e, team.OrgID, team.ID, userID)
 	if err != nil || !isMember {
 		return err
@@ -558,6 +560,16 @@ func removeTeamMember(e Engine, team *Team, userID int64) error {
 		}
 	}
 
+	// Check if the user is a member of any team in the organization.
+	if count, err := e.Count(&TeamUser{
+		UID:   userID,
+		OrgID: team.OrgID,
+	}); err != nil {
+		return err
+	} else if count == 0 {
+		return removeOrgUser(e, team.OrgID, userID)
+	}
+
 	return nil
 }
 

models/release.go

@@ -36,7 +36,7 @@ type Release struct {
 	IsPrerelease     bool           `xorm:"NOT NULL DEFAULT false"`
 	IsTag            bool           `xorm:"NOT NULL DEFAULT false"`
 	Attachments      []*Attachment  `xorm:"-"`
-	CreatedUnix      util.TimeStamp `xorm:"created INDEX"`
+	CreatedUnix      util.TimeStamp `xorm:"INDEX"`
 }
 
 func (r *Release) loadAttributes(e Engine) error {
@@ -134,6 +134,8 @@ func createTag(gitRepo *git.Repository, rel *Release) error {
 		if err != nil {
 			return fmt.Errorf("CommitsCount: %v", err)
 		}
+	} else {
+		rel.CreatedUnix = util.TimeStampNow()
 	}
 	return nil
 }

models/repo.go

@@ -163,6 +163,7 @@ func NewRepoContext() {
 type Repository struct {
 	ID            int64  `xorm:"pk autoincr"`
 	OwnerID       int64  `xorm:"UNIQUE(s)"`
+	OwnerName     string `xorm:"-"`
 	Owner         *User  `xorm:"-"`
 	LowerName     string `xorm:"UNIQUE(s) INDEX NOT NULL"`
 	Name          string `xorm:"INDEX NOT NULL"`
@@ -223,9 +224,17 @@ func (repo *Repository) MustOwner() *User {
 	return repo.mustOwner(x)
 }
 
+// MustOwnerName always returns valid owner name to avoid
+// conceptually impossible error handling.
+// It returns "error" and logs error details when error
+// occurs.
+func (repo *Repository) MustOwnerName() string {
+	return repo.mustOwnerName(x)
+}
+
 // FullName returns the repository full name
 func (repo *Repository) FullName() string {
-	return repo.MustOwner().Name + "/" + repo.Name
+	return repo.MustOwnerName() + "/" + repo.Name
 }
 
 // HTMLURL returns the repository HTML URL
@@ -477,6 +486,41 @@ func (repo *Repository) mustOwner(e Engine) *User {
 	return repo.Owner
 }
 
+func (repo *Repository) getOwnerName(e Engine) error {
+	if len(repo.OwnerName) > 0 {
+		return nil
+	}
+
+	if repo.Owner != nil {
+		repo.OwnerName = repo.Owner.Name
+		return nil
+	}
+
+	u := new(User)
+	has, err := e.ID(repo.OwnerID).Cols("name").Get(u)
+	if err != nil {
+		return err
+	} else if !has {
+		return ErrUserNotExist{repo.OwnerID, "", 0}
+	}
+	repo.OwnerName = u.Name
+	return nil
+}
+
+// GetOwnerName returns the repository owner name
+func (repo *Repository) GetOwnerName() error {
+	return repo.getOwnerName(x)
+}
+
+func (repo *Repository) mustOwnerName(e Engine) string {
+	if err := repo.getOwnerName(e); err != nil {
+		log.Error(4, "Error loading repository owner name: %v", err)
+		return "error"
+	}
+
+	return repo.OwnerName
+}
+
 // ComposeMetas composes a map of metas for rendering external issue tracker URL.
 func (repo *Repository) ComposeMetas() map[string]string {
 	unit, err := repo.GetUnit(UnitTypeExternalTracker)
@@ -588,7 +632,7 @@ func (repo *Repository) GetBaseRepo() (err error) {
 }
 
 func (repo *Repository) repoPath(e Engine) string {
-	return RepoPath(repo.mustOwner(e).Name, repo.Name)
+	return RepoPath(repo.mustOwnerName(e), repo.Name)
 }
 
 // RepoPath returns the repository path
@@ -1131,7 +1175,7 @@ type CreateRepoOptions struct {
 }
 
 func getRepoInitFile(tp, name string) ([]byte, error) {
-	cleanedName := strings.TrimLeft(name, "./")
+	cleanedName := strings.TrimLeft(path.Clean("/"+name), "/")
 	relPath := path.Join("options", tp, cleanedName)
 
 	// Use custom file when available.
@@ -1778,6 +1822,8 @@ func DeleteRepository(doer *User, uid, repoID int64) error {
 		&PullRequest{BaseRepoID: repoID},
 		&RepoUnit{RepoID: repoID},
 		&RepoRedirect{RedirectRepoID: repoID},
+		&Webhook{RepoID: repoID},
+		&HookTask{RepoID: repoID},
 	); err != nil {
 		return fmt.Errorf("deleteBeans: %v", err)
 	}
@@ -1800,6 +1846,9 @@ func DeleteRepository(doer *User, uid, repoID int64) error {
 		if _, err = sess.In("issue_id", issueIDs).Delete(&IssueUser{}); err != nil {
 			return err
 		}
+		if _, err = sess.In("issue_id", issueIDs).Delete(&Reaction{}); err != nil {
+			return err
+		}
 
 		attachments := make([]*Attachment, 0, 5)
 		if err = sess.
@@ -2133,7 +2182,7 @@ func ReinitMissingRepositories() error {
 // SyncRepositoryHooks rewrites all repositories' pre-receive, update and post-receive hooks
 // to make sure the binary and custom conf path are up-to-date.
 func SyncRepositoryHooks() error {
-	return x.Where("id > 0").Iterate(new(Repository),
+	return x.Cols("owner_id", "name").Where("id > 0").Iterate(new(Repository),
 		func(idx int, bean interface{}) error {
 			if err := createDelegateHooks(bean.(*Repository).RepoPath()); err != nil {
 				return fmt.Errorf("SyncRepositoryHook: %v", err)

models/repo_activity.go

@@ -176,7 +176,7 @@ func (stats *ActivityStats) FillIssues(repoID int64, fromTime time.Time) error {
 
 	// Closed issues
 	sess := issuesForActivityStatement(repoID, fromTime, true, false)
-	sess.OrderBy("issue.updated_unix DESC")
+	sess.OrderBy("issue.closed_unix DESC")
 	stats.ClosedIssues = make(IssueList, 0)
 	if err = sess.Find(&stats.ClosedIssues); err != nil {
 		return err
@@ -228,7 +228,11 @@ func issuesForActivityStatement(repoID int64, fromTime time.Time, closed, unreso
 
 	if !unresolved {
 		sess.And("issue.is_pull = ?", false)
-		sess.And("issue.created_unix >= ?", fromTime.Unix())
+		if closed {
+			sess.And("issue.closed_unix >= ?", fromTime.Unix())
+		} else {
+			sess.And("issue.created_unix >= ?", fromTime.Unix())
+		}
 	} else {
 		sess.And("issue.created_unix < ?", fromTime.Unix())
 		sess.And("issue.updated_unix >= ?", fromTime.Unix())

models/repo_indexer.go

@@ -5,9 +5,7 @@
 package models
 
 import (
-	"io/ioutil"
-	"os"
-	"path"
+	"fmt"
 	"strconv"
 	"strings"
 
@@ -16,8 +14,6 @@ import (
 	"code.gitea.io/gitea/modules/indexer"
 	"code.gitea.io/gitea/modules/log"
 	"code.gitea.io/gitea/modules/setting"
-
-	"github.com/Unknwon/com"
 )
 
 // RepoIndexerStatus status of a repo's entry in the repo indexer
@@ -132,7 +128,11 @@ func populateRepoIndexer(maxRepoID int64) {
 }
 
 func updateRepoIndexer(repo *Repository) error {
-	changes, err := getRepoChanges(repo)
+	sha, err := getDefaultBranchSha(repo)
+	if err != nil {
+		return err
+	}
+	changes, err := getRepoChanges(repo, sha)
 	if err != nil {
 		return err
 	} else if changes == nil {
@@ -140,12 +140,12 @@ func updateRepoIndexer(repo *Repository) error {
 	}
 
 	batch := indexer.RepoIndexerBatch()
-	for _, filename := range changes.UpdatedFiles {
-		if err := addUpdate(filename, repo, batch); err != nil {
+	for _, update := range changes.Updates {
+		if err := addUpdate(update, repo, batch); err != nil {
 			return err
 		}
 	}
-	for _, filename := range changes.RemovedFiles {
+	for _, filename := range changes.RemovedFilenames {
 		if err := addDelete(filename, repo, batch); err != nil {
 			return err
 		}
@@ -153,56 +153,61 @@ func updateRepoIndexer(repo *Repository) error {
 	if err = batch.Flush(); err != nil {
 		return err
 	}
-	return updateLastIndexSync(repo)
+	return repo.updateIndexerStatus(sha)
 }
 
 // repoChanges changes (file additions/updates/removals) to a repo
 type repoChanges struct {
-	UpdatedFiles []string
-	RemovedFiles []string
+	Updates          []fileUpdate
+	RemovedFilenames []string
+}
+
+type fileUpdate struct {
+	Filename string
+	BlobSha  string
+}
+
+func getDefaultBranchSha(repo *Repository) (string, error) {
+	stdout, err := git.NewCommand("show-ref", "-s", repo.DefaultBranch).RunInDir(repo.RepoPath())
+	if err != nil {
+		return "", err
+	}
+	return strings.TrimSpace(stdout), nil
 }
 
 // getRepoChanges returns changes to repo since last indexer update
-func getRepoChanges(repo *Repository) (*repoChanges, error) {
-	repoWorkingPool.CheckIn(com.ToStr(repo.ID))
-	defer repoWorkingPool.CheckOut(com.ToStr(repo.ID))
-
-	if err := repo.UpdateLocalCopyBranch(""); err != nil {
-		return nil, err
-	} else if !git.IsBranchExist(repo.LocalCopyPath(), repo.DefaultBranch) {
-		// repo does not have any commits yet, so nothing to update
-		return nil, nil
-	} else if err = repo.UpdateLocalCopyBranch(repo.DefaultBranch); err != nil {
-		return nil, err
-	} else if err = repo.getIndexerStatus(); err != nil {
+func getRepoChanges(repo *Repository, revision string) (*repoChanges, error) {
+	if err := repo.getIndexerStatus(); err != nil {
 		return nil, err
 	}
 
 	if len(repo.IndexerStatus.CommitSha) == 0 {
-		return genesisChanges(repo)
+		return genesisChanges(repo, revision)
 	}
-	return nonGenesisChanges(repo)
+	return nonGenesisChanges(repo, revision)
 }
 
-func addUpdate(filename string, repo *Repository, batch *indexer.Batch) error {
-	filepath := path.Join(repo.LocalCopyPath(), filename)
-	if stat, err := os.Stat(filepath); err != nil {
+func addUpdate(update fileUpdate, repo *Repository, batch *indexer.Batch) error {
+	stdout, err := git.NewCommand("cat-file", "-s", update.BlobSha).
+		RunInDir(repo.RepoPath())
+	if err != nil {
 		return err
-	} else if stat.Size() > setting.Indexer.MaxIndexerFileSize {
-		return nil
-	} else if stat.IsDir() {
-		// file could actually be a directory, if it is the root of a submodule.
-		// We do not index submodule contents, so don't do anything.
+	}
+	if size, err := strconv.Atoi(strings.TrimSpace(stdout)); err != nil {
+		return fmt.Errorf("Misformatted git cat-file output: %v", err)
+	} else if int64(size) > setting.Indexer.MaxIndexerFileSize {
 		return nil
 	}
-	fileContents, err := ioutil.ReadFile(filepath)
+
+	fileContents, err := git.NewCommand("cat-file", "blob", update.BlobSha).
+		RunInDirBytes(repo.RepoPath())
 	if err != nil {
 		return err
 	} else if !base.IsTextFile(fileContents) {
 		return nil
 	}
 	return batch.Add(indexer.RepoIndexerUpdate{
-		Filepath: filename,
+		Filepath: update.Filename,
 		Op:       indexer.RepoIndexerOpUpdate,
 		Data: &indexer.RepoIndexerData{
 			RepoID:  repo.ID,
@@ -221,42 +226,76 @@ func addDelete(filename string, repo *Repository, batch *indexer.Batch) error {
 	})
 }
 
-// genesisChanges get changes to add repo to the indexer for the first time
-func genesisChanges(repo *Repository) (*repoChanges, error) {
-	var changes repoChanges
-	stdout, err := git.NewCommand("ls-files").RunInDir(repo.LocalCopyPath())
-	if err != nil {
-		return nil, err
-	}
-	for _, line := range strings.Split(stdout, "\n") {
-		filename := strings.TrimSpace(line)
-		if len(filename) == 0 {
+// parseGitLsTreeOutput parses the output of a `git ls-tree -r --full-name` command
+func parseGitLsTreeOutput(stdout string) ([]fileUpdate, error) {
+	lines := strings.Split(stdout, "\n")
+	updates := make([]fileUpdate, 0, len(lines))
+	for _, line := range lines {
+		// expect line to be "<mode> <object-type> <object-sha>\t<filename>"
+		line = strings.TrimSpace(line)
+		if len(line) == 0 {
 			continue
-		} else if filename[0] == '"' {
+		}
+		firstSpaceIndex := strings.IndexByte(line, ' ')
+		if firstSpaceIndex < 0 {
+			log.Error(4, "Misformatted git ls-tree output: %s", line)
+			continue
+		}
+		tabIndex := strings.IndexByte(line, '\t')
+		if tabIndex < 42+firstSpaceIndex || tabIndex == len(line)-1 {
+			log.Error(4, "Misformatted git ls-tree output: %s", line)
+			continue
+		}
+		if objectType := line[firstSpaceIndex+1 : tabIndex-41]; objectType != "blob" {
+			// submodules appear as commit objects, we do not index submodules
+			continue
+		}
+
+		blobSha := line[tabIndex-40 : tabIndex]
+		filename := line[tabIndex+1:]
+		if filename[0] == '"' {
+			var err error
 			filename, err = strconv.Unquote(filename)
 			if err != nil {
 				return nil, err
 			}
 		}
-		changes.UpdatedFiles = append(changes.UpdatedFiles, filename)
+		updates = append(updates, fileUpdate{
+			Filename: filename,
+			BlobSha:  blobSha,
+		})
 	}
-	return &changes, nil
+	return updates, nil
+}
+
+// genesisChanges get changes to add repo to the indexer for the first time
+func genesisChanges(repo *Repository, revision string) (*repoChanges, error) {
+	var changes repoChanges
+	stdout, err := git.NewCommand("ls-tree", "--full-tree", "-r", revision).
+		RunInDir(repo.RepoPath())
+	if err != nil {
+		return nil, err
+	}
+	changes.Updates, err = parseGitLsTreeOutput(stdout)
+	return &changes, err
 }
 
 // nonGenesisChanges get changes since the previous indexer update
-func nonGenesisChanges(repo *Repository) (*repoChanges, error) {
+func nonGenesisChanges(repo *Repository, revision string) (*repoChanges, error) {
 	diffCmd := git.NewCommand("diff", "--name-status",
-		repo.IndexerStatus.CommitSha, "HEAD")
-	stdout, err := diffCmd.RunInDir(repo.LocalCopyPath())
+		repo.IndexerStatus.CommitSha, revision)
+	stdout, err := diffCmd.RunInDir(repo.RepoPath())
 	if err != nil {
 		// previous commit sha may have been removed by a force push, so
 		// try rebuilding from scratch
+		log.Warn("git diff: %v", err)
 		if err = indexer.DeleteRepoFromIndexer(repo.ID); err != nil {
 			return nil, err
 		}
-		return genesisChanges(repo)
+		return genesisChanges(repo, revision)
 	}
 	var changes repoChanges
+	updatedFilenames := make([]string, 0, 10)
 	for _, line := range strings.Split(stdout, "\n") {
 		line = strings.TrimSpace(line)
 		if len(line) == 0 {
@@ -274,23 +313,22 @@ func nonGenesisChanges(repo *Repository) (*repoChanges, error) {
 
 		switch status := line[0]; status {
 		case 'M', 'A':
-			changes.UpdatedFiles = append(changes.UpdatedFiles, filename)
+			updatedFilenames = append(updatedFilenames, filename)
 		case 'D':
-			changes.RemovedFiles = append(changes.RemovedFiles, filename)
+			changes.RemovedFilenames = append(changes.RemovedFilenames, filename)
 		default:
 			log.Warn("Unrecognized status: %c (line=%s)", status, line)
 		}
 	}
-	return &changes, nil
-}
 
-func updateLastIndexSync(repo *Repository) error {
-	stdout, err := git.NewCommand("rev-parse", "HEAD").RunInDir(repo.LocalCopyPath())
+	cmd := git.NewCommand("ls-tree", "--full-tree", revision, "--")
+	cmd.AddArguments(updatedFilenames...)
+	stdout, err = cmd.RunInDir(repo.RepoPath())
 	if err != nil {
-		return err
+		return nil, err
 	}
-	sha := strings.TrimSpace(stdout)
-	return repo.updateIndexerStatus(sha)
+	changes.Updates, err = parseGitLsTreeOutput(stdout)
+	return &changes, err
 }
 
 func processRepoIndexerOperationQueue() {

models/repo_mirror.go

@@ -244,6 +244,8 @@ func MirrorUpdate() {
 // SyncMirrors checks and syncs mirrors.
 // TODO: sync more mirrors at same time.
 func SyncMirrors() {
+	sess := x.NewSession()
+	defer sess.Close()
 	// Start listening on new sync requests.
 	for repoID := range MirrorQueue.Queue() {
 		log.Trace("SyncMirrors [repo_id: %v]", repoID)
@@ -260,10 +262,22 @@ func SyncMirrors() {
 		}
 
 		m.ScheduleNextUpdate()
-		if err = UpdateMirror(m); err != nil {
+		if err = updateMirror(sess, m); err != nil {
 			log.Error(4, "UpdateMirror [%s]: %v", repoID, err)
 			continue
 		}
+
+		// Get latest commit date and update to current repository updated time
+		commitDate, err := git.GetLatestCommitTime(m.Repo.RepoPath())
+		if err != nil {
+			log.Error(2, "GetLatestCommitDate [%s]: %v", m.RepoID, err)
+			continue
+		}
+
+		if _, err = sess.Exec("UPDATE repository SET updated_unix = ? WHERE id = ?", commitDate.Unix(), m.RepoID); err != nil {
+			log.Error(2, "Update repository 'updated_unix' [%s]: %v", m.RepoID, err)
+			continue
+		}
 	}
 }
 

models/unit_tests.go

@@ -9,6 +9,7 @@ import (
 	"os"
 	"path/filepath"
 	"testing"
+	"time"
 
 	"code.gitea.io/gitea/modules/setting"
 
@@ -72,6 +73,18 @@ func createTestEngine(fixturesDir string) error {
 	return InitFixtures(&testfixtures.SQLite{}, fixturesDir)
 }
 
+func removeAllWithRetry(dir string) error {
+	var err error
+	for i := 0; i < 20; i++ {
+		err = os.RemoveAll(dir)
+		if err == nil {
+			break
+		}
+		time.Sleep(100 * time.Millisecond)
+	}
+	return err
+}
+
 // PrepareTestDatabase load test fixtures into test database
 func PrepareTestDatabase() error {
 	return LoadFixtures()
@@ -81,7 +94,7 @@ func PrepareTestDatabase() error {
 // by tests that use the above MainTest(..) function.
 func PrepareTestEnv(t testing.TB) {
 	assert.NoError(t, PrepareTestDatabase())
-	assert.NoError(t, os.RemoveAll(setting.RepoRootPath))
+	assert.NoError(t, removeAllWithRetry(setting.RepoRootPath))
 	metaPath := filepath.Join(giteaRoot, "integrations", "gitea-repositories-meta")
 	assert.NoError(t, com.CopyDir(metaPath, setting.RepoRootPath))
 }

models/user.go

@@ -299,7 +299,9 @@ func (u *User) generateRandomAvatar(e Engine) error {
 	}
 	// NOTICE for random avatar, it still uses id as avatar name, but custom avatar use md5
 	// since random image is not a user's photo, there is no security for enumable
-	u.Avatar = fmt.Sprintf("%d", u.ID)
+	if u.Avatar == "" {
+		u.Avatar = fmt.Sprintf("%d", u.ID)
+	}
 	if err = os.MkdirAll(filepath.Dir(u.CustomAvatarPath()), os.ModePerm); err != nil {
 		return fmt.Errorf("MkdirAll: %v", err)
 	}
@@ -649,7 +651,7 @@ func NewGhostUser() *User {
 }
 
 var (
-	reservedUsernames    = []string{"assets", "css", "explore", "img", "js", "less", "plugins", "debug", "raw", "install", "api", "avatars", "user", "org", "help", "stars", "issues", "pulls", "commits", "repo", "template", "admin", "new", ".", ".."}
+	reservedUsernames    = []string{"assets", "css", "explore", "img", "js", "less", "plugins", "debug", "raw", "install", "api", "avatars", "user", "org", "help", "stars", "issues", "pulls", "commits", "repo", "template", "admin", "error", "new", ".", ".."}
 	reservedUserPatterns = []string{"*.keys"}
 )
 
@@ -932,7 +934,7 @@ func deleteUser(e *xorm.Session, u *User) error {
 	if err = e.Table("star").Cols("star.repo_id").
 		Where("star.uid = ?", u.ID).Find(&starredRepoIDs); err != nil {
 		return fmt.Errorf("get all stars: %v", err)
-	} else if _, err = e.Decr("num_watches").In("id", starredRepoIDs).Update(new(Repository)); err != nil {
+	} else if _, err = e.Decr("num_stars").In("id", starredRepoIDs).Update(new(Repository)); err != nil {
 		return fmt.Errorf("decrease repository num_stars: %v", err)
 	}
 	// ***** END: Star *****

models/webhook.go

@@ -437,7 +437,14 @@ func (t *HookTask) AfterLoad() {
 
 	t.RequestInfo = &HookRequest{}
 	if err := json.Unmarshal([]byte(t.RequestContent), t.RequestInfo); err != nil {
-		log.Error(3, "Unmarshal[%d]: %v", t.ID, err)
+		log.Error(3, "Unmarshal RequestContent[%d]: %v", t.ID, err)
+	}
+
+	if len(t.ResponseContent) > 0 {
+		t.ResponseInfo = &HookResponse{}
+		if err := json.Unmarshal([]byte(t.ResponseContent), t.ResponseInfo); err != nil {
+			log.Error(3, "Unmarshal ResponseContent[%d]: %v", t.ID, err)
+		}
 	}
 }
 
@@ -619,6 +626,10 @@ func (t *HookTask) deliver() {
 			log.Trace("Hook delivery failed: %s", t.UUID)
 		}
 
+		if err := UpdateHookTask(t); err != nil {
+			log.Error(4, "UpdateHookTask [%d]: %v", t.ID, err)
+		}
+
 		// Update webhook last delivery status.
 		w, err := GetWebhookByID(t.HookID)
 		if err != nil {
@@ -671,10 +682,6 @@ func DeliverHooks() {
 	// Update hook task status.
 	for _, t := range tasks {
 		t.deliver()
-
-		if err := UpdateHookTask(t); err != nil {
-			log.Error(4, "UpdateHookTask [%d]: %v", t.ID, err)
-		}
 	}
 
 	// Start listening on new hook requests.
@@ -695,10 +702,6 @@ func DeliverHooks() {
 		}
 		for _, t := range tasks {
 			t.deliver()
-			if err := UpdateHookTask(t); err != nil {
-				log.Error(4, "UpdateHookTask [%d]: %v", t.ID, err)
-				continue
-			}
 		}
 	}
 }

models/wiki.go

@@ -45,7 +45,7 @@ func WikiNameToFilename(name string) string {
 // WikiFilenameToName converts a wiki filename to its corresponding page name.
 func WikiFilenameToName(filename string) (string, error) {
 	if !strings.HasSuffix(filename, ".md") {
-		return "", fmt.Errorf("Invalid wiki filename: %s", filename)
+		return "", ErrWikiInvalidFileName{filename}
 	}
 	basename := filename[:len(filename)-3]
 	unescaped, err := url.QueryUnescape(basename)
@@ -67,7 +67,7 @@ func WikiPath(userName, repoName string) string {
 
 // WikiPath returns wiki data path for given repository.
 func (repo *Repository) WikiPath() string {
-	return WikiPath(repo.MustOwner().Name, repo.Name)
+	return WikiPath(repo.MustOwnerName(), repo.Name)
 }
 
 // HasWiki returns true if repository has wiki.

models/wiki_test.go

@@ -77,11 +77,14 @@ func TestWikiFilenameToName(t *testing.T) {
 	for _, badFilename := range []string{
 		"nofileextension",
 		"wrongfileextension.txt",
-		"badescaping%%.md",
 	} {
 		_, err := WikiFilenameToName(badFilename)
 		assert.Error(t, err)
+		assert.True(t, IsErrWikiInvalidFileName(err))
 	}
+	_, err := WikiFilenameToName("badescaping%%.md")
+	assert.Error(t, err)
+	assert.False(t, IsErrWikiInvalidFileName(err))
 }
 
 func TestWikiNameToFilenameToName(t *testing.T) {

modules/auth/user_form.go

@@ -182,7 +182,7 @@ func (f *AddKeyForm) Validate(ctx *macaron.Context, errs binding.Errors) binding
 
 // NewAccessTokenForm form for creating access token
 type NewAccessTokenForm struct {
-	Name string `binding:"Required"`
+	Name string `binding:"Required;MaxSize(255)"`
 }
 
 // Validate valideates the fields

modules/context/context.go

@@ -9,6 +9,7 @@ import (
 	"html/template"
 	"io"
 	"net/http"
+	"net/url"
 	"path"
 	"strings"
 	"time"
@@ -75,6 +76,26 @@ func (ctx *Context) HasValue(name string) bool {
 	return ok
 }
 
+// RedirectToFirst redirects to first not empty URL
+func (ctx *Context) RedirectToFirst(location ...string) {
+	for _, loc := range location {
+		if len(loc) == 0 {
+			continue
+		}
+
+		u, err := url.Parse(loc)
+		if err != nil || (u.Scheme != "" && !strings.HasPrefix(strings.ToLower(loc), strings.ToLower(setting.AppURL))) {
+			continue
+		}
+
+		ctx.Redirect(loc)
+		return
+	}
+
+	ctx.Redirect(setting.AppSubURL + "/")
+	return
+}
+
 // HTML calls Context.HTML and converts template name to string.
 func (ctx *Context) HTML(status int, name base.TplName) {
 	log.Debug("Template: %s", name)

modules/lfs/server.go

@@ -83,6 +83,8 @@ type link struct {
 	ExpiresAt time.Time         `json:"expires_at,omitempty"`
 }
 
+var oidRegExp = regexp.MustCompile(`^[A-Fa-f0-9]+$`)
+
 // ObjectOidHandler is the main request routing entry point into LFS server functions
 func ObjectOidHandler(ctx *context.Context) {
 
@@ -217,6 +219,12 @@ func PostHandler(ctx *context.Context) {
 
 	if !authenticate(ctx, repository, rv.Authorization, true) {
 		requireAuth(ctx)
+		return
+	}
+
+	if !oidRegExp.MatchString(rv.Oid) {
+		writeStatus(ctx, 404)
+		return
 	}
 
 	meta, err := models.NewLFSMetaObject(&models.LFSMetaObject{Oid: rv.Oid, Size: rv.Size, RepositoryID: repository.ID})
@@ -284,10 +292,12 @@ func BatchHandler(ctx *context.Context) {
 			continue
 		}
 
-		// Object is not found
-		meta, err = models.NewLFSMetaObject(&models.LFSMetaObject{Oid: object.Oid, Size: object.Size, RepositoryID: repository.ID})
-		if err == nil {
-			responseObjects = append(responseObjects, Represent(object, meta, meta.Existing, !contentStore.Exists(meta)))
+		if oidRegExp.MatchString(object.Oid) {
+			// Object is not found
+			meta, err = models.NewLFSMetaObject(&models.LFSMetaObject{Oid: object.Oid, Size: object.Size, RepositoryID: repository.ID})
+			if err == nil {
+				responseObjects = append(responseObjects, Represent(object, meta, meta.Existing, !contentStore.Exists(meta)))
+			}
 		}
 	}
 

modules/markup/html.go

@@ -114,16 +114,25 @@ func cutoutVerbosePrefix(prefix string) string {
 
 // URLJoin joins url components, like path.Join, but preserving contents
 func URLJoin(base string, elems ...string) string {
-	u, err := url.Parse(base)
+	if !strings.HasSuffix(base, "/") {
+		base += "/"
+	}
+	baseURL, err := url.Parse(base)
 	if err != nil {
 		log.Error(4, "URLJoin: Invalid base URL %s", base)
 		return ""
 	}
-	joinArgs := make([]string, 0, len(elems)+1)
-	joinArgs = append(joinArgs, u.Path)
-	joinArgs = append(joinArgs, elems...)
-	u.Path = path.Join(joinArgs...)
-	return u.String()
+	joinedPath := path.Join(elems...)
+	argURL, err := url.Parse(joinedPath)
+	if err != nil {
+		log.Error(4, "URLJoin: Invalid arg %s", joinedPath)
+		return ""
+	}
+	joinedURL := baseURL.ResolveReference(argURL).String()
+	if !baseURL.IsAbs() && !strings.HasPrefix(base, "/") {
+		return joinedURL[1:] // Removing leading '/' if needed
+	}
+	return joinedURL
 }
 
 // RenderIssueIndexPatternOptions options for RenderIssueIndexPattern function
@@ -391,7 +400,14 @@ func RenderShortLinks(rawBytes []byte, urlPrefix string, noLink bool, isWikiMark
 		}
 		absoluteLink := isLink([]byte(link))
 		if !absoluteLink {
-			link = strings.Replace(link, " ", "+", -1)
+			if image {
+				link = strings.Replace(link, " ", "+", -1)
+			} else {
+				link = strings.Replace(link, " ", "-", -1)
+			}
+			if !strings.Contains(link, "/") {
+				link = url.PathEscape(link)
+			}
 		}
 		if image {
 			if !absoluteLink {

modules/markup/html_test.go

@@ -83,6 +83,14 @@ func TestURLJoin(t *testing.T) {
 			"a", "b/c/"),
 		newTest("a/b/d",
 			"a/", "b/c/", "/../d/"),
+		newTest("https://try.gitea.io/a/b/c#d",
+			"https://try.gitea.io", "a/b", "c#d"),
+		newTest("/a/b/d",
+			"/a/", "b/c/", "/../d/"),
+		newTest("/a/b/c",
+			"/a", "b/c/"),
+		newTest("/a/b/c#hash",
+			"/a", "b/c#hash"),
 	} {
 		assert.Equal(t, test.Expected, URLJoin(test.Base, test.Elements...))
 	}

modules/markup/markdown/markdown_test.go

@@ -55,10 +55,16 @@ func TestRender_ShortLinks(t *testing.T) {
 	rawtree := markup.URLJoin(AppSubURL, "raw", "master")
 	url := markup.URLJoin(tree, "Link")
 	otherUrl := markup.URLJoin(tree, "OtherLink")
+	encodedURL := markup.URLJoin(tree, "Link%3F")
 	imgurl := markup.URLJoin(rawtree, "Link.jpg")
+	encodedImgurl := markup.URLJoin(rawtree, "Link+%23.jpg")
+	notencodedImgurl := markup.URLJoin(rawtree, "some", "path", "Link+#.jpg")
 	urlWiki := markup.URLJoin(AppSubURL, "wiki", "Link")
 	otherUrlWiki := markup.URLJoin(AppSubURL, "wiki", "OtherLink")
+	encodedURLWiki := markup.URLJoin(AppSubURL, "wiki", "Link%3F")
 	imgurlWiki := markup.URLJoin(AppSubURL, "wiki", "raw", "Link.jpg")
+	encodedImgurlWiki := markup.URLJoin(AppSubURL, "wiki", "raw", "Link+%23.jpg")
+	notencodedImgurlWiki := markup.URLJoin(AppSubURL, "wiki", "raw", "some", "path", "Link+#.jpg")
 	favicon := "http://google.com/favicon.ico"
 
 	test(
@@ -101,6 +107,26 @@ func TestRender_ShortLinks(t *testing.T) {
 		"[[Link]] [[OtherLink]]",
 		`<p><a href="`+url+`" rel="nofollow">Link</a> <a href="`+otherUrl+`" rel="nofollow">OtherLink</a></p>`,
 		`<p><a href="`+urlWiki+`" rel="nofollow">Link</a> <a href="`+otherUrlWiki+`" rel="nofollow">OtherLink</a></p>`)
+	test(
+		"[[Link?]]",
+		`<p><a href="`+encodedURL+`" rel="nofollow">Link?</a></p>`,
+		`<p><a href="`+encodedURLWiki+`" rel="nofollow">Link?</a></p>`)
+	test(
+		"[[Link]] [[OtherLink]] [[Link?]]",
+		`<p><a href="`+url+`" rel="nofollow">Link</a> <a href="`+otherUrl+`" rel="nofollow">OtherLink</a> <a href="`+encodedURL+`" rel="nofollow">Link?</a></p>`,
+		`<p><a href="`+urlWiki+`" rel="nofollow">Link</a> <a href="`+otherUrlWiki+`" rel="nofollow">OtherLink</a> <a href="`+encodedURLWiki+`" rel="nofollow">Link?</a></p>`)
+	test(
+		"[[Link #.jpg]]",
+		`<p><a href="`+encodedImgurl+`" rel="nofollow"><img src="`+encodedImgurl+`"/></a></p>`,
+		`<p><a href="`+encodedImgurlWiki+`" rel="nofollow"><img src="`+encodedImgurlWiki+`"/></a></p>`)
+	test(
+		"[[Name|Link #.jpg|alt=\"AltName\"|title='Title']]",
+		`<p><a href="`+encodedImgurl+`" rel="nofollow"><img src="`+encodedImgurl+`" alt="AltName" title="Title"/></a></p>`,
+		`<p><a href="`+encodedImgurlWiki+`" rel="nofollow"><img src="`+encodedImgurlWiki+`" alt="AltName" title="Title"/></a></p>`)
+	test(
+		"[[some/path/Link #.jpg]]",
+		`<p><a href="`+notencodedImgurl+`" rel="nofollow"><img src="`+notencodedImgurl+`"/></a></p>`,
+		`<p><a href="`+notencodedImgurlWiki+`" rel="nofollow"><img src="`+notencodedImgurlWiki+`"/></a></p>`)
 }
 
 func TestMisc_IsMarkdownFile(t *testing.T) {

modules/setting/setting.go

@@ -956,7 +956,7 @@ func NewContext() {
 	AttachmentAllowedTypes = strings.Replace(sec.Key("ALLOWED_TYPES").MustString("image/jpeg,image/png,application/zip,application/gzip"), "|", ",", -1)
 	AttachmentMaxSize = sec.Key("MAX_SIZE").MustInt64(4)
 	AttachmentMaxFiles = sec.Key("MAX_FILES").MustInt(5)
-	AttachmentEnabled = sec.Key("ENABLE").MustBool(true)
+	AttachmentEnabled = sec.Key("ENABLED").MustBool(true)
 
 	TimeFormatKey := Cfg.Section("time").Key("FORMAT").MustString("RFC1123")
 	TimeFormat = map[string]string{

modules/templates/helper.go

@@ -10,6 +10,7 @@ import (
 	"encoding/json"
 	"errors"
 	"fmt"
+	"html"
 	"html/template"
 	"mime"
 	"net/url"
@@ -179,6 +180,7 @@ func NewFuncMap() []template.FuncMap {
 			return dict, nil
 		},
 		"Printf": fmt.Sprintf,
+		"Escape": Escape,
 	}}
 }
 
@@ -197,6 +199,11 @@ func Str2html(raw string) template.HTML {
 	return template.HTML(markup.Sanitize(raw))
 }
 
+// Escape escapes a HTML string
+func Escape(raw string) string {
+	return html.EscapeString(raw)
+}
+
 // List traversings the list
 func List(l *list.List) chan interface{} {
 	e := l.Front()

modules/util/time_stamp.go

@@ -59,3 +59,8 @@ func (ts TimeStamp) FormatLong() string {
 func (ts TimeStamp) FormatShort() string {
 	return ts.Format("Jan 02, 2006")
 }
+
+// IsZero is zero time
+func (ts TimeStamp) IsZero() bool {
+	return ts.AsTime().IsZero()
+}

modules/util/util.go

@@ -4,6 +4,15 @@
 
 package util
 
+import (
+	"net/url"
+	"path"
+	"strings"
+
+	"code.gitea.io/gitea/modules/log"
+	"code.gitea.io/gitea/modules/setting"
+)
+
 // OptionalBool a boolean that can be "null"
 type OptionalBool byte
 
@@ -47,6 +56,41 @@ func Max(a, b int) int {
 	return a
 }
 
+// URLJoin joins url components, like path.Join, but preserving contents
+func URLJoin(base string, elems ...string) string {
+	if !strings.HasSuffix(base, "/") {
+		base += "/"
+	}
+	baseURL, err := url.Parse(base)
+	if err != nil {
+		log.Error(4, "URLJoin: Invalid base URL %s", base)
+		return ""
+	}
+	joinedPath := path.Join(elems...)
+	argURL, err := url.Parse(joinedPath)
+	if err != nil {
+		log.Error(4, "URLJoin: Invalid arg %s", joinedPath)
+		return ""
+	}
+	joinedURL := baseURL.ResolveReference(argURL).String()
+	if !baseURL.IsAbs() && !strings.HasPrefix(base, "/") {
+		return joinedURL[1:] // Removing leading '/' if needed
+	}
+	return joinedURL
+}
+
+// IsExternalURL checks if rawURL points to an external URL like http://example.com
+func IsExternalURL(rawURL string) bool {
+	parsed, err := url.Parse(rawURL)
+	if err != nil {
+		return true
+	}
+	if len(parsed.Host) != 0 && strings.Replace(parsed.Host, "www.", "", 1) != strings.Replace(setting.Domain, "www.", "", 1) {
+		return true
+	}
+	return false
+}
+
 // Min min of two ints
 func Min(a, b int) int {
 	if a > b {

modules/util/util_test.go

@@ -0,0 +1,79 @@
+// Copyright 2018 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package util
+
+import (
+	"testing"
+
+	"code.gitea.io/gitea/modules/setting"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestURLJoin(t *testing.T) {
+	type test struct {
+		Expected string
+		Base     string
+		Elements []string
+	}
+	newTest := func(expected, base string, elements ...string) test {
+		return test{Expected: expected, Base: base, Elements: elements}
+	}
+	for _, test := range []test{
+		newTest("https://try.gitea.io/a/b/c",
+			"https://try.gitea.io", "a/b", "c"),
+		newTest("https://try.gitea.io/a/b/c",
+			"https://try.gitea.io/", "/a/b/", "/c/"),
+		newTest("https://try.gitea.io/a/c",
+			"https://try.gitea.io/", "/a/./b/", "../c/"),
+		newTest("a/b/c",
+			"a", "b/c/"),
+		newTest("a/b/d",
+			"a/", "b/c/", "/../d/"),
+		newTest("https://try.gitea.io/a/b/c#d",
+			"https://try.gitea.io", "a/b", "c#d"),
+		newTest("/a/b/d",
+			"/a/", "b/c/", "/../d/"),
+		newTest("/a/b/c",
+			"/a", "b/c/"),
+		newTest("/a/b/c#hash",
+			"/a", "b/c#hash"),
+	} {
+		assert.Equal(t, test.Expected, URLJoin(test.Base, test.Elements...))
+	}
+}
+
+func TestIsExternalURL(t *testing.T) {
+	setting.Domain = "try.gitea.io"
+	type test struct {
+		Expected bool
+		RawURL   string
+	}
+	newTest := func(expected bool, rawURL string) test {
+		return test{Expected: expected, RawURL: rawURL}
+	}
+	for _, test := range []test{
+		newTest(false,
+			"https://try.gitea.io"),
+		newTest(true,
+			"https://example.com/"),
+		newTest(true,
+			"//example.com"),
+		newTest(true,
+			"http://example.com"),
+		newTest(false,
+			"a/"),
+		newTest(false,
+			"https://try.gitea.io/test?param=false"),
+		newTest(false,
+			"test?param=false"),
+		newTest(false,
+			"//try.gitea.io/test?param=false"),
+		newTest(false,
+			"/hey/hey/hey#3244"),
+	} {
+		assert.Equal(t, test.Expected, IsExternalURL(test.RawURL))
+	}
+}

public/less/_base.less

@@ -152,6 +152,10 @@ pre, code {
         }
     }
 
+    &.floating.label {
+        z-index: 10;
+    }
+
     &.menu,
     &.vertical.menu,
     &.segment {
@@ -167,6 +171,14 @@ pre, code {
         font-size: .92857143rem;
     }
 
+    &.dropdown .menu>.item>.floating.label {
+        z-index: 11;
+    }
+
+    &.dropdown .menu .menu>.item>.floating.label {
+        z-index: 21;
+    }
+
     .text {
         &.red {
             color: #d95c5c !important;

routers/repo/editor.go

@@ -163,7 +163,7 @@ func editFilePost(ctx *context.Context, form auth.EditRepoFileForm, isNewFile bo
 		branchName = form.NewBranchName
 	}
 
-	form.TreePath = strings.Trim(form.TreePath, " /")
+	form.TreePath = strings.Trim(path.Clean("/"+form.TreePath), " /")
 	treeNames, treePaths := getParentTreeFields(form.TreePath)
 
 	ctx.Data["TreePath"] = form.TreePath
@@ -477,7 +477,7 @@ func UploadFilePost(ctx *context.Context, form auth.UploadRepoFileForm) {
 		branchName = form.NewBranchName
 	}
 
-	form.TreePath = strings.Trim(form.TreePath, " /")
+	form.TreePath = strings.Trim(path.Clean("/"+form.TreePath), " /")
 	treeNames, treePaths := getParentTreeFields(form.TreePath)
 	if len(treeNames) == 0 {
 		// We must at least have one element for user to input.

routers/repo/http.go

@@ -184,33 +184,33 @@ func HTTP(ctx *context.Context) {
 					return
 				}
 			}
+		}
 
-			if !isPublicPull {
-				has, err := models.HasAccess(authUser.ID, repo, accessMode)
-				if err != nil {
-					ctx.ServerError("HasAccess", err)
-					return
-				} else if !has {
-					if accessMode == models.AccessModeRead {
-						has, err = models.HasAccess(authUser.ID, repo, models.AccessModeWrite)
-						if err != nil {
-							ctx.ServerError("HasAccess2", err)
-							return
-						} else if !has {
-							ctx.HandleText(http.StatusForbidden, "User permission denied")
-							return
-						}
-					} else {
+		if !isPublicPull {
+			has, err := models.HasAccess(authUser.ID, repo, accessMode)
+			if err != nil {
+				ctx.ServerError("HasAccess", err)
+				return
+			} else if !has {
+				if accessMode == models.AccessModeRead {
+					has, err = models.HasAccess(authUser.ID, repo, models.AccessModeWrite)
+					if err != nil {
+						ctx.ServerError("HasAccess2", err)
+						return
+					} else if !has {
 						ctx.HandleText(http.StatusForbidden, "User permission denied")
 						return
 					}
-				}
-
-				if !isPull && repo.IsMirror {
-					ctx.HandleText(http.StatusForbidden, "mirror repository is read-only")
+				} else {
+					ctx.HandleText(http.StatusForbidden, "User permission denied")
 					return
 				}
 			}
+
+			if !isPull && repo.IsMirror {
+				ctx.HandleText(http.StatusForbidden, "mirror repository is read-only")
+				return
+			}
 		}
 
 		if !repo.CheckUnitUser(authUser.ID, authUser.IsAdmin, unitType) {

routers/repo/issue_label_test.go

@@ -32,7 +32,7 @@ func TestInitializeLabels(t *testing.T) {
 	ctx := test.MockContext(t, "user2/repo1/labels/initialize")
 	test.LoadUser(t, ctx, 2)
 	test.LoadRepo(t, ctx, 2)
-	InitializeLabels(ctx, auth.InitializeLabelsForm{"Default"})
+	InitializeLabels(ctx, auth.InitializeLabelsForm{TemplateName: "Default"})
 	assert.EqualValues(t, http.StatusFound, ctx.Resp.Status())
 	models.AssertExistsAndLoadBean(t, &models.Label{
 		RepoID: 2,

routers/repo/repo.go

@@ -307,11 +307,7 @@ func Action(ctx *context.Context) {
 		return
 	}
 
-	redirectTo := ctx.Query("redirect_to")
-	if len(redirectTo) == 0 {
-		redirectTo = ctx.Repo.RepoLink
-	}
-	ctx.Redirect(redirectTo)
+	ctx.RedirectToFirst(ctx.Query("redirect_to"), ctx.Repo.RepoLink)
 }
 
 // Download download an archive of a repository

routers/repo/view.go

@@ -105,7 +105,9 @@ func renderDirectory(ctx *context.Context, treeLink string) {
 					ctx.Data["FileContent"] = string(markup.Render(readmeFile.Name(), buf, treeLink, ctx.Repo.Repository.ComposeMetas()))
 				} else {
 					ctx.Data["IsRenderedHTML"] = true
-					ctx.Data["FileContent"] = string(bytes.Replace(buf, []byte("\n"), []byte(`<br>`), -1))
+					ctx.Data["FileContent"] = strings.Replace(
+						gotemplate.HTMLEscapeString(string(buf)), "\n", `<br>`, -1,
+					)
 				}
 			}
 		}
@@ -208,7 +210,9 @@ func renderFile(ctx *context.Context, entry *git.TreeEntry, treeLink, rawLink st
 			ctx.Data["FileContent"] = string(markup.Render(blob.Name(), buf, path.Dir(treeLink), ctx.Repo.Repository.ComposeMetas()))
 		} else if readmeExist {
 			ctx.Data["IsRenderedHTML"] = true
-			ctx.Data["FileContent"] = string(bytes.Replace(buf, []byte("\n"), []byte(`<br>`), -1))
+			ctx.Data["FileContent"] = strings.Replace(
+				gotemplate.HTMLEscapeString(string(buf)), "\n", `<br>`, -1,
+			)
 		} else {
 			// Building code view blocks with line number on server side.
 			var fileContent string
@@ -223,6 +227,10 @@ func renderFile(ctx *context.Context, entry *git.TreeEntry, treeLink, rawLink st
 
 			var output bytes.Buffer
 			lines := strings.Split(fileContent, "\n")
+			//Remove blank line at the end of file
+			if len(lines) > 0 && lines[len(lines)-1] == "" {
+				lines = lines[:len(lines)-1]
+			}
 			for index, line := range lines {
 				line = gotemplate.HTMLEscapeString(line)
 				if index != len(lines)-1 {

routers/repo/webhook.go

@@ -205,7 +205,7 @@ func GogsHooksNewPost(ctx *context.Context, form auth.NewGogshookForm) {
 		Secret:       form.Secret,
 		HookEvent:    ParseHookEvent(form.WebhookForm),
 		IsActive:     form.Active,
-		HookTaskType: models.GITEA,
+		HookTaskType: models.GOGS,
 		OrgID:        orCtx.OrgID,
 	}
 	if err := w.UpdateEvent(); err != nil {

routers/repo/wiki.go

@@ -128,6 +128,9 @@ func renderWikiPage(ctx *context.Context, isViewPage bool) (*git.Repository, *gi
 			}
 			wikiName, err := models.WikiFilenameToName(entry.Name())
 			if err != nil {
+				if models.IsErrWikiInvalidFileName(err) {
+					continue
+				}
 				ctx.ServerError("WikiFilenameToName", err)
 				return nil, nil
 			} else if wikiName == "_Sidebar" || wikiName == "_Footer" {
@@ -262,6 +265,9 @@ func WikiPages(ctx *context.Context) {
 		}
 		wikiName, err := models.WikiFilenameToName(entry.Name())
 		if err != nil {
+			if models.IsErrWikiInvalidFileName(err) {
+				continue
+			}
 			ctx.ServerError("WikiFilenameToName", err)
 			return
 		}
@@ -344,7 +350,7 @@ func NewWikiPost(ctx *context.Context, form auth.NewWikiForm) {
 		return
 	}
 
-	ctx.Redirect(ctx.Repo.RepoLink + "/wiki/" + models.WikiNameToFilename(wikiName))
+	ctx.Redirect(ctx.Repo.RepoLink + "/wiki/" + models.WikiNameToSubURL(wikiName))
 }
 
 // EditWiki render wiki modify page
@@ -385,7 +391,7 @@ func EditWikiPost(ctx *context.Context, form auth.NewWikiForm) {
 		return
 	}
 
-	ctx.Redirect(ctx.Repo.RepoLink + "/wiki/" + models.WikiNameToFilename(newWikiName))
+	ctx.Redirect(ctx.Repo.RepoLink + "/wiki/" + models.WikiNameToSubURL(newWikiName))
 }
 
 // DeleteWikiPagePost delete wiki page

routers/repo/wiki_test.go

@@ -7,36 +7,52 @@ package repo
 import (
 	"io/ioutil"
 	"net/http"
-	"path/filepath"
 	"testing"
 
+	"code.gitea.io/git"
 	"code.gitea.io/gitea/models"
 	"code.gitea.io/gitea/modules/auth"
 	"code.gitea.io/gitea/modules/test"
 
-	"github.com/Unknwon/com"
 	"github.com/stretchr/testify/assert"
 )
 
 const content = "Wiki contents for unit tests"
 const message = "Wiki commit message for unit tests"
 
-func wikiPath(repo *models.Repository, wikiName string) string {
-	return filepath.Join(repo.LocalWikiPath(), models.WikiNameToFilename(wikiName))
+func wikiEntry(t *testing.T, repo *models.Repository, wikiName string) *git.TreeEntry {
+	wikiRepo, err := git.OpenRepository(repo.WikiPath())
+	assert.NoError(t, err)
+	commit, err := wikiRepo.GetBranchCommit("master")
+	assert.NoError(t, err)
+	entries, err := commit.ListEntries()
+	assert.NoError(t, err)
+	for _, entry := range entries {
+		if entry.Name() == models.WikiNameToFilename(wikiName) {
+			return entry
+		}
+	}
+	return nil
 }
 
 func wikiContent(t *testing.T, repo *models.Repository, wikiName string) string {
-	bytes, err := ioutil.ReadFile(wikiPath(repo, wikiName))
+	entry := wikiEntry(t, repo, wikiName)
+	if !assert.NotNil(t, entry) {
+		return ""
+	}
+	reader, err := entry.Blob().Data()
+	assert.NoError(t, err)
+	bytes, err := ioutil.ReadAll(reader)
 	assert.NoError(t, err)
 	return string(bytes)
 }
 
 func assertWikiExists(t *testing.T, repo *models.Repository, wikiName string) {
-	assert.True(t, com.IsExist(wikiPath(repo, wikiName)))
+	assert.NotNil(t, wikiEntry(t, repo, wikiName))
 }
 
 func assertWikiNotExists(t *testing.T, repo *models.Repository, wikiName string) {
-	assert.False(t, com.IsExist(wikiPath(repo, wikiName)))
+	assert.Nil(t, wikiEntry(t, repo, wikiName))
 }
 
 func assertPagesMetas(t *testing.T, expectedNames []string, metas interface{}) {

routers/routes/routes.go

@@ -446,7 +446,7 @@ func RegisterRoutes(m *macaron.Macaron) {
 				m.Get("/:id", repo.WebHooksEdit)
 				m.Post("/:id/test", repo.TestWebhook)
 				m.Post("/gitea/:id", bindIgnErr(auth.NewWebhookForm{}), repo.WebHooksEditPost)
-				m.Post("/gogs/:id", bindIgnErr(auth.NewGogshookForm{}), repo.GogsHooksNewPost)
+				m.Post("/gogs/:id", bindIgnErr(auth.NewGogshookForm{}), repo.GogsHooksEditPost)
 				m.Post("/slack/:id", bindIgnErr(auth.NewSlackHookForm{}), repo.SlackHooksEditPost)
 				m.Post("/discord/:id", bindIgnErr(auth.NewDiscordHookForm{}), repo.DiscordHooksEditPost)
 				m.Post("/dingtalk/:id", bindIgnErr(auth.NewDingtalkHookForm{}), repo.DingtalkHooksEditPost)

routers/user/auth.go

@@ -18,6 +18,7 @@ import (
 	"code.gitea.io/gitea/modules/context"
 	"code.gitea.io/gitea/modules/log"
 	"code.gitea.io/gitea/modules/setting"
+	"code.gitea.io/gitea/modules/util"
 
 	"github.com/go-macaron/captcha"
 	"github.com/markbates/goth"
@@ -93,12 +94,8 @@ func checkAutoLogin(ctx *context.Context) bool {
 	}
 
 	if isSucceed {
-		if len(redirectTo) > 0 {
-			ctx.SetCookie("redirect_to", "", -1, setting.AppSubURL)
-			ctx.Redirect(redirectTo)
-		} else {
-			ctx.Redirect(setting.AppSubURL + string(setting.LandingPageURL))
-		}
+		ctx.SetCookie("redirect_to", "", -1, setting.AppSubURL)
+		ctx.RedirectToFirst(redirectTo, setting.AppSubURL+string(setting.LandingPageURL))
 		return true
 	}
 
@@ -347,10 +344,10 @@ func handleSignInFull(ctx *context.Context, u *models.User, remember bool, obeyR
 		return
 	}
 
-	if redirectTo, _ := url.QueryUnescape(ctx.GetCookie("redirect_to")); len(redirectTo) > 0 {
+	if redirectTo, _ := url.QueryUnescape(ctx.GetCookie("redirect_to")); len(redirectTo) > 0 && !util.IsExternalURL(redirectTo) {
 		ctx.SetCookie("redirect_to", "", -1, setting.AppSubURL)
 		if obeyRedirect {
-			ctx.Redirect(redirectTo)
+			ctx.RedirectToFirst(redirectTo)
 		}
 		return
 	}
@@ -439,7 +436,7 @@ func handleOAuth2SignIn(u *models.User, gothUser goth.User, ctx *context.Context
 
 			if redirectTo, _ := url.QueryUnescape(ctx.GetCookie("redirect_to")); len(redirectTo) > 0 {
 				ctx.SetCookie("redirect_to", "", -1, setting.AppSubURL)
-				ctx.Redirect(redirectTo)
+				ctx.RedirectToFirst(redirectTo)
 				return
 			}
 

routers/user/auth_openid.go

@@ -49,12 +49,8 @@ func SignInOpenID(ctx *context.Context) {
 	}
 
 	if isSucceed {
-		if len(redirectTo) > 0 {
-			ctx.SetCookie("redirect_to", "", -1, setting.AppSubURL)
-			ctx.Redirect(redirectTo)
-		} else {
-			ctx.Redirect(setting.AppSubURL + "/")
-		}
+		ctx.SetCookie("redirect_to", "", -1, setting.AppSubURL)
+		ctx.RedirectToFirst(redirectTo)
 		return
 	}
 

routers/user/home.go

@@ -66,12 +66,14 @@ func retrieveFeeds(ctx *context.Context, options models.GetFeedsOptions) {
 	if ctx.User != nil {
 		userCache[ctx.User.ID] = ctx.User
 	}
-	repoCache := map[int64]*models.Repository{}
 	for _, act := range actions {
-		// Cache results to reduce queries.
-		u, ok := userCache[act.ActUserID]
+		if act.ActUser != nil {
+			userCache[act.ActUserID] = act.ActUser
+		}
+
+		repoOwner, ok := userCache[act.Repo.OwnerID]
 		if !ok {
-			u, err = models.GetUserByID(act.ActUserID)
+			repoOwner, err = models.GetUserByID(act.Repo.OwnerID)
 			if err != nil {
 				if models.IsErrUserNotExist(err) {
 					continue
@@ -79,35 +81,9 @@ func retrieveFeeds(ctx *context.Context, options models.GetFeedsOptions) {
 				ctx.ServerError("GetUserByID", err)
 				return
 			}
-			userCache[act.ActUserID] = u
+			userCache[repoOwner.ID] = repoOwner
 		}
-		act.ActUser = u
-
-		repo, ok := repoCache[act.RepoID]
-		if !ok {
-			repo, err = models.GetRepositoryByID(act.RepoID)
-			if err != nil {
-				if models.IsErrRepoNotExist(err) {
-					continue
-				}
-				ctx.ServerError("GetRepositoryByID", err)
-				return
-			}
-		}
-		act.Repo = repo
-
-		repoOwner, ok := userCache[repo.OwnerID]
-		if !ok {
-			repoOwner, err = models.GetUserByID(repo.OwnerID)
-			if err != nil {
-				if models.IsErrUserNotExist(err) {
-					continue
-				}
-				ctx.ServerError("GetUserByID", err)
-				return
-			}
-		}
-		repo.Owner = repoOwner
+		act.Repo.Owner = repoOwner
 	}
 	ctx.Data["Feeds"] = actions
 }
@@ -154,7 +130,8 @@ func Dashboard(ctx *context.Context) {
 	ctx.Data["MirrorCount"] = len(mirrors)
 	ctx.Data["Mirrors"] = mirrors
 
-	retrieveFeeds(ctx, models.GetFeedsOptions{RequestedUser: ctxUser,
+	retrieveFeeds(ctx, models.GetFeedsOptions{
+		RequestedUser:   ctxUser,
 		IncludePrivate:  true,
 		OnlyPerformedBy: false,
 		IncludeDeleted:  false,

routers/user/profile.go

@@ -271,9 +271,5 @@ func Action(ctx *context.Context) {
 		return
 	}
 
-	redirectTo := ctx.Query("redirect_to")
-	if len(redirectTo) == 0 {
-		redirectTo = u.HomeLink()
-	}
-	ctx.Redirect(redirectTo)
+	ctx.RedirectToFirst(ctx.Query("redirect_to"), u.HomeLink())
 }

templates/admin/monitor.tmpl

@@ -49,8 +49,8 @@
 						<tr>
 							<td>{{.PID}}</td>
 							<td>{{.Description}}</td>
-							<td>{{.Start.FormatLong}}</td>
-							<td>{{TimeSinceUnix .Start $.Lang}}</td>
+							<td>{{DateFmtLong .Start}}</td>
+							<td>{{TimeSince .Start $.Lang}}</td>
 						</tr>
 					{{end}}
 				</tbody>

templates/repo/activity.tmpl

@@ -134,7 +134,7 @@
 					<p class="desc">
 						<div class="ui red label">{{$.i18n.Tr "repo.activity.closed_issue_label"}}</div>
 						#{{.Index}} <a class="title has-emoji" href="{{$.Repository.HTMLURL}}/issues/{{.Index}}">{{.Title}}</a>
-						{{TimeSinceUnix .UpdatedUnix $.Lang}}
+						{{TimeSinceUnix .ClosedUnix $.Lang}}
 					</p>
 				{{end}}
 			</div>

templates/repo/bare.tmpl

@@ -5,7 +5,7 @@
 		<div class="ui grid">
 			<div class="sixteen wide column content">
 				{{template "base/alert" .}}
-				{{if .IsRepositoryAdmin}}
+				{{if .IsRepositoryWriter}}
 					<h4 class="ui top attached header">
 						{{.i18n.Tr "repo.quick_guide"}}
 					</h4>

templates/repo/branch/list.tmpl

@@ -69,12 +69,12 @@
 <div class="ui small basic delete modal">
 	<div class="ui icon header">
 		<i class="trash icon"></i>
-		{{.i18n.Tr "repo.branch.delete_html"| Safe}} <span class="branch-name"></span>
+		{{.i18n.Tr "repo.branch.delete_html"}} <span class="branch-name"></span>
 	</div>
 	<div class="content">
-		<p>{{.i18n.Tr "repo.branch.delete_desc" | Safe}}</p>
+		<p>{{.i18n.Tr "repo.branch.delete_desc"}}</p>
 		{{.i18n.Tr "repo.branch.delete_notices_1" | Safe}}<br>
-		{{.i18n.Tr "repo.branch.delete_notices_html" | Safe}} <span class="branch-name"></span><br>
+		{{.i18n.Tr "repo.branch.delete_notices_html"}} <span class="branch-name"></span><br>
 	</div>
 	{{template "base/delete_modal_actions" .}}
 </div>

templates/repo/branch_dropdown.tmpl

@@ -47,9 +47,9 @@
 						</div>
 						<div class="text small">
 							{{if .IsViewBranch}}
-								{{.i18n.Tr "repo.branch.create_from" .BranchName | Safe}}
+								{{.i18n.Tr "repo.branch.create_from" .BranchName}}
 							{{else}}
-								{{.i18n.Tr "repo.branch.create_from" (ShortSha .BranchName) | Safe}}
+								{{.i18n.Tr "repo.branch.create_from" (ShortSha .BranchName)}}
 							{{end}}
 						</div>
 					</a>

templates/repo/create.tmpl

@@ -36,7 +36,7 @@
 					<div class="inline required field {{if .Err_RepoName}}error{{end}}">
 						<label for="repo_name">{{.i18n.Tr "repo.repo_name"}}</label>
 						<input id="repo_name" name="repo_name" value="{{.repo_name}}" autofocus required>
-						<span class="help">{{.i18n.Tr "repo.repo_name_helper" | Safe}}</span>
+						<span class="help">{{.i18n.Tr "repo.repo_name_helper"}}</span>
 					</div>
 					<div class="inline field">
 						<label>{{.i18n.Tr "repo.visibility"}}</label>

templates/repo/editor/commit_form.tmpl

@@ -14,8 +14,7 @@
 					<input type="radio" class="js-quick-pull-choice-option" name="commit_choice" value="direct" {{if eq .commit_choice "direct"}}checked{{end}}>
 					<label>
 						<i class="octicon octicon-git-commit" height="16" width="14"></i>
-						{{$branchName := .BranchName | Str2html}}
-						{{.i18n.Tr "repo.editor.commit_directly_to_this_branch" $branchName | Safe}}
+						{{.i18n.Tr "repo.editor.commit_directly_to_this_branch" (.BranchName|Escape) | Safe}}
 					</label>
 				</div>
 			</div>

templates/repo/header.tmpl

@@ -48,7 +48,7 @@
 	<div class="ui tabs container">
 		<div class="ui tabular stackable menu navbar">
 			{{if .Repository.UnitEnabled $.UnitTypeCode}}
-			<a class="{{if .PageIsViewCode}}active{{end}} item" href="{{.RepoLink}}">
+			<a class="{{if .PageIsViewCode}}active{{end}} item" href="{{.RepoLink}}{{if (ne .BranchName .Repository.DefaultBranch)}}/src/{{.BranchNameSubURL}}{{end}}">
 				<i class="octicon octicon-code"></i> {{.i18n.Tr "repo.code"}}
 			</a>
 			{{end}}

templates/repo/home.tmpl

@@ -73,7 +73,7 @@
 						{{else if and (not $.DisableSSH) (or $.IsSigned $.ExposeAnonSSH)}}
 							<input id="repo-clone-url" value="{{$.CloneLink.SSH}}" readonly>
 						{{end}}
-						{{if or ((not $.DisableHTTP) (and (not $.DisableSSH) (or $.IsSigned $.ExposeAnonSSH)))}}
+						{{if or (not $.DisableHTTP) (and (not $.DisableSSH) (or $.IsSigned $.ExposeAnonSSH))}}
 							<button class="ui basic icon button poping up clipboard" id="clipboard-btn" data-original="{{.i18n.Tr "repo.copy_link"}}" data-success="{{.i18n.Tr "repo.copy_link_success"}}" data-error="{{.i18n.Tr "repo.copy_link_error"}}" data-content="{{.i18n.Tr "repo.copy_link"}}" data-variation="inverted tiny" data-clipboard-target="#repo-clone-url">
 								<i class="octicon octicon-clippy"></i>
 							</button>

templates/repo/issue/view_content.tmpl

@@ -134,12 +134,12 @@
 <div class="ui small basic delete modal">
 	<div class="ui icon header">
 		<i class="trash icon"></i>
-		{{.i18n.Tr "repo.branch.delete" .HeadTarget | Safe}}
+		{{.i18n.Tr "repo.branch.delete" .HeadTarget }}
 	</div>
 	<div class="content">
-		<p>{{.i18n.Tr "repo.branch.delete_desc" | Safe}}</p>
+		<p>{{.i18n.Tr "repo.branch.delete_desc"}}</p>
 		{{.i18n.Tr "repo.branch.delete_notices_1" | Safe}}<br>
-		{{.i18n.Tr "repo.branch.delete_notices_2" .HeadTarget | Safe}}<br>
+		{{.i18n.Tr "repo.branch.delete_notices_2" .HeadTarget}}<br>
 	</div>
 	{{template "base/delete_modal_actions" .}}
 </div>

templates/repo/issue/view_content/comments.tmpl

@@ -103,7 +103,7 @@
 					<img src="{{.Poster.RelAvatarLink}}">
 				</a>
 				<span class="text grey"><a href="{{.Poster.HomeLink}}">{{.Poster.Name}}</a>
-				{{if .Content}}{{$.i18n.Tr "repo.issues.add_label_at" .Label.ForegroundColor .Label.Color .Label.Name $createdStr | Safe}}{{else}}{{$.i18n.Tr "repo.issues.remove_label_at" .Label.ForegroundColor .Label.Color .Label.Name $createdStr | Safe}}{{end}}</span>
+				{{if .Content}}{{$.i18n.Tr "repo.issues.add_label_at" .Label.ForegroundColor .Label.Color (.Label.Name|Escape) $createdStr | Safe}}{{else}}{{$.i18n.Tr "repo.issues.remove_label_at" .Label.ForegroundColor .Label.Color (.Label.Name|Escape) $createdStr | Safe}}{{end}}</span>
 			</div>
 		{{end}}
 	{{else if eq .Type 8}}
@@ -113,7 +113,7 @@
 				<img src="{{.Poster.RelAvatarLink}}">
 			</a>
 			<span class="text grey"><a href="{{.Poster.HomeLink}}">{{.Poster.Name}}</a>
-			{{if gt .OldMilestoneID 0}}{{if gt .MilestoneID 0}}{{$.i18n.Tr "repo.issues.change_milestone_at" .OldMilestone.Name .Milestone.Name $createdStr | Safe}}{{else}}{{$.i18n.Tr "repo.issues.remove_milestone_at" .OldMilestone.Name $createdStr | Safe}}{{end}}{{else if gt .MilestoneID 0}}{{$.i18n.Tr "repo.issues.add_milestone_at" .Milestone.Name $createdStr | Safe}}{{end}}</span>
+			{{if gt .OldMilestoneID 0}}{{if gt .MilestoneID 0}}{{$.i18n.Tr "repo.issues.change_milestone_at" (.OldMilestone.Name|Escape) (.Milestone.Name|Escape) $createdStr | Safe}}{{else}}{{$.i18n.Tr "repo.issues.remove_milestone_at" (.OldMilestone.Name|Escape) $createdStr | Safe}}{{end}}{{else if gt .MilestoneID 0}}{{$.i18n.Tr "repo.issues.add_milestone_at" (.Milestone.Name|Escape) $createdStr | Safe}}{{end}}</span>
 		</div>
 	{{else if eq .Type 9}}
 		<div class="event">
@@ -131,23 +131,23 @@
 	{{else if eq .Type 10}}
 		<div class="event">
 			<span class="octicon octicon-primitive-dot"></span>
+			<a class="ui avatar image" href="{{.Poster.HomeLink}}">
+				<img src="{{.Poster.RelAvatarLink}}">
+			</a>
+			<span class="text grey"><a href="{{.Poster.HomeLink}}">{{.Poster.Name}}</a>
+			{{$.i18n.Tr "repo.issues.change_title_at" (.OldTitle|Escape) (.NewTitle|Escape) $createdStr | Safe}}
+			</span>
 		</div>
-		<a class="ui avatar image" href="{{.Poster.HomeLink}}">
-			<img src="{{.Poster.RelAvatarLink}}">
-		</a>
-		<span class="text grey"><a href="{{.Poster.HomeLink}}">{{.Poster.Name}}</a>
-		{{$.i18n.Tr "repo.issues.change_title_at" .OldTitle .NewTitle $createdStr | Safe}}
-		</span>
 	{{else if eq .Type 11}}
 		<div class="event">
 			<span class="octicon octicon-primitive-dot"></span>
+			<a class="ui avatar image" href="{{.Poster.HomeLink}}">
+				<img src="{{.Poster.RelAvatarLink}}">
+			</a>
+			<span class="text grey"><a href="{{.Poster.HomeLink}}">{{.Poster.Name}}</a>
+			{{$.i18n.Tr "repo.issues.delete_branch_at" (.CommitSHA|Escape) $createdStr | Safe}}
+			</span>
 		</div>
-		<a class="ui avatar image" href="{{.Poster.HomeLink}}">
-			<img src="{{.Poster.RelAvatarLink}}">
-		</a>
-		<span class="text grey"><a href="{{.Poster.HomeLink}}">{{.Poster.Name}}</a>
-		{{$.i18n.Tr "repo.issues.delete_branch_at" .CommitSHA $createdStr | Safe}}
-		</span>
     {{else if eq .Type 12}}
 		<div class="event">
 			<span class="octicon octicon-primitive-dot"></span>

templates/repo/search.tmpl

@@ -14,7 +14,7 @@
 		</div>
 		{{if .Keyword}}
 			<h3>
-				{{.i18n.Tr "repo.search.results" .Keyword .RepoLink .RepoName | Str2html}}
+				{{.i18n.Tr "repo.search.results" (.Keyword|Escape) .RepoLink .RepoName | Str2html }}
 			</h3>
 			<div class="repository search">
 				{{range $result := .SearchResults}}

templates/repo/settings/options.tmpl

@@ -302,7 +302,7 @@
 		</div>
 		<div class="content">
 			<div class="ui warning message text left">
-				{{.i18n.Tr "repo.settings.convert_notices_1" | Safe}}
+				{{.i18n.Tr "repo.settings.convert_notices_1"}}
 			</div>
 			<form class="ui form" action="{{.Link}}" method="post">
 				{{.CsrfTokenHtml}}
@@ -333,8 +333,8 @@
 		</div>
 		<div class="content">
 			<div class="ui warning message text left">
-				{{.i18n.Tr "repo.settings.transfer_notices_1" | Safe}} <br>
-				{{.i18n.Tr "repo.settings.transfer_notices_2" | Safe}}
+				{{.i18n.Tr "repo.settings.transfer_notices_1"}} <br>
+				{{.i18n.Tr "repo.settings.transfer_notices_2"}}
 			</div>
 			<form class="ui form" action="{{.Link}}" method="post">
 				{{.CsrfTokenHtml}}
@@ -371,7 +371,7 @@
 				{{.i18n.Tr "repo.settings.delete_notices_1" | Safe}}<br>
 				{{.i18n.Tr "repo.settings.delete_notices_2" .Repository.FullName | Safe}}
 				{{if .Repository.NumForks}}<br>
-				{{.i18n.Tr "repo.settings.delete_notices_fork_1" | Safe}}
+				{{.i18n.Tr "repo.settings.delete_notices_fork_1"}}
 				{{end}}
 			</div>
 			<form class="ui form" action="{{.Link}}" method="post">

templates/repo/wiki/view.tmpl

@@ -104,7 +104,7 @@
 		{{.i18n.Tr "repo.wiki.delete_page_button"}}
 	</div>
 	<div class="content">
-		<p>{{.i18n.Tr "repo.wiki.delete_page_notice_1" $title | Safe}}</p>
+		<p>{{.i18n.Tr "repo.wiki.delete_page_notice_1" ($title|Escape) | Safe}}</p>
 	</div>
 	{{template "base/delete_modal_actions" .}}
 </div>

templates/user/settings/keys_gpg.tmpl

@@ -26,7 +26,7 @@
 					<div class="activity meta">
 						<i>{{$.i18n.Tr "settings.add_on"}} <span>{{.AddedUnix.FormatShort}}</span></i>
 						-
-						<i>{{if .ExpiredUnix}}{{$.i18n.Tr "settings.valid_until"}} <span>{{.ExpiredUnix.FormatShort}}</span>{{else}}{{$.i18n.Tr "settings.valid_forever"}}{{end}}</i>
+						<i>{{if not .ExpiredUnix.IsZero}}{{$.i18n.Tr "settings.valid_until"}} <span>{{.ExpiredUnix.FormatShort}}</span>{{else}}{{$.i18n.Tr "settings.valid_forever"}}{{end}}</i>
 					</div>
 				</div>
 			</div>

vendor/code.gitea.io/git/Gopkg.lock

@@ -0,0 +1,38 @@
+# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
+
+
+[[projects]]
+  branch = "master"
+  name = "github.com/Unknwon/com"
+  packages = ["."]
+  revision = "7677a1d7c1137cd3dd5ba7a076d0c898a1ef4520"
+
+[[projects]]
+  name = "github.com/davecgh/go-spew"
+  packages = ["spew"]
+  revision = "6d212800a42e8ab5c146b8ace3490ee17e5225f9"
+
+[[projects]]
+  name = "github.com/mcuadros/go-version"
+  packages = ["."]
+  revision = "257f7b9a7d87427c8d7f89469a5958d57f8abd7c"
+
+[[projects]]
+  name = "github.com/pmezard/go-difflib"
+  packages = ["difflib"]
+  revision = "d8ed2627bdf02c080bf22230dbb337003b7aba2d"
+
+[[projects]]
+  name = "github.com/stretchr/testify"
+  packages = [
+    "assert",
+    "require"
+  ]
+  revision = "976c720a22c8eb4eb6a0b4348ad85ad12491a506"
+
+[solve-meta]
+  analyzer-name = "dep"
+  analyzer-version = 1
+  inputs-digest = "d37e90051cd58dd1f99f808626e82d64eac47f2b2334c6fcb9179bfcf2814622"
+  solver-name = "gps-cdcl"
+  solver-version = 1

vendor/code.gitea.io/git/Gopkg.toml

@@ -0,0 +1,34 @@
+# Gopkg.toml
+#
+# Refer to https://golang.github.io/dep/docs/Gopkg.toml.html
+# for detailed Gopkg.toml documentation.
+#
+# required = ["github.com/user/thing/cmd/thing"]
+# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
+#
+# [[constraint]]
+#   name = "github.com/user/project"
+#   version = "1.0.0"
+#
+# [[constraint]]
+#   name = "github.com/user/project2"
+#   branch = "dev"
+#   source = "github.com/myfork/project2"
+#
+# [[override]]
+#   name = "github.com/x/y"
+#   version = "2.4.0"
+#
+# [prune]
+#   non-go = false
+#   go-tests = true
+#   unused-packages = true
+
+
+[[constraint]]
+  name = "github.com/Unknwon/com"
+  branch = "master"
+
+[prune]
+  go-tests = true
+  unused-packages = true

vendor/code.gitea.io/git/MAINTAINERS

@@ -18,3 +18,4 @@ Antoine Girard <sapk@sapk.fr> (@sapk)
 Jonas Östanbäck <jonas.ostanback@gmail.com> (@cez81)
 David Schneiderbauer <dschneiderbauer@gmail.com> (@daviian)
 Peter Žeby <morlinest@gmail.com> (@morlinest)
+Jonas Franz <info@jonasfranz.software> (@JonasFranzDEV)

vendor/code.gitea.io/git/error.go

@@ -40,6 +40,16 @@ func (err ErrNotExist) Error() string {
 	return fmt.Sprintf("object does not exist [id: %s, rel_path: %s]", err.ID, err.RelPath)
 }
 
+// ErrBadLink entry.FollowLink error
+type ErrBadLink struct {
+	Name    string
+	Message string
+}
+
+func (err ErrBadLink) Error() string {
+	return fmt.Sprintf("%s: %s", err.Name, err.Message)
+}
+
 // ErrUnsupportedVersion error when required git version not matched
 type ErrUnsupportedVersion struct {
 	Required string

vendor/code.gitea.io/git/parse.go

@@ -0,0 +1,81 @@
+// Copyright 2018 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package git
+
+import (
+	"bytes"
+	"fmt"
+	"strconv"
+)
+
+// ParseTreeEntries parses the output of a `git ls-tree` command.
+func ParseTreeEntries(data []byte) ([]*TreeEntry, error) {
+	return parseTreeEntries(data, nil)
+}
+
+func parseTreeEntries(data []byte, ptree *Tree) ([]*TreeEntry, error) {
+	entries := make([]*TreeEntry, 0, 10)
+	for pos := 0; pos < len(data); {
+		// expect line to be of the form "<mode> <type> <sha>\t<filename>"
+		entry := new(TreeEntry)
+		entry.ptree = ptree
+		if pos+6 > len(data) {
+			return nil, fmt.Errorf("Invalid ls-tree output: %s", string(data))
+		}
+		switch string(data[pos : pos+6]) {
+		case "100644":
+			entry.mode = EntryModeBlob
+			entry.Type = ObjectBlob
+			pos += 12 // skip over "100644 blob "
+		case "100755":
+			entry.mode = EntryModeExec
+			entry.Type = ObjectBlob
+			pos += 12 // skip over "100755 blob "
+		case "120000":
+			entry.mode = EntryModeSymlink
+			entry.Type = ObjectBlob
+			pos += 12 // skip over "120000 blob "
+		case "160000":
+			entry.mode = EntryModeCommit
+			entry.Type = ObjectCommit
+			pos += 14 // skip over "160000 object "
+		case "040000":
+			entry.mode = EntryModeTree
+			entry.Type = ObjectTree
+			pos += 12 // skip over "040000 tree "
+		default:
+			return nil, fmt.Errorf("unknown type: %v", string(data[pos:pos+6]))
+		}
+
+		if pos+40 > len(data) {
+			return nil, fmt.Errorf("Invalid ls-tree output: %s", string(data))
+		}
+		id, err := NewIDFromString(string(data[pos : pos+40]))
+		if err != nil {
+			return nil, fmt.Errorf("Invalid ls-tree output: %v", err)
+		}
+		entry.ID = id
+		pos += 41 // skip over sha and trailing space
+
+		end := pos + bytes.IndexByte(data[pos:], '\n')
+		if end < pos {
+			return nil, fmt.Errorf("Invalid ls-tree output: %s", string(data))
+		}
+
+		// In case entry name is surrounded by double quotes(it happens only in git-shell).
+		if data[pos] == '"' {
+			entry.name, err = strconv.Unquote(string(data[pos:end]))
+			if err != nil {
+				return nil, fmt.Errorf("Invalid ls-tree output: %v", err)
+			}
+		} else {
+			entry.name = string(data[pos:end])
+		}
+
+		pos = end + 1
+		entries = append(entries, entry)
+	}
+	return entries, nil
+}

vendor/code.gitea.io/git/repo_blame.go

@@ -4,7 +4,21 @@
 
 package git
 
+import "fmt"
+
 // FileBlame return the Blame object of file
 func (repo *Repository) FileBlame(revision, path, file string) ([]byte, error) {
-	return NewCommand("blame", "--root", file).RunInDirBytes(path)
+	return NewCommand("blame", "--root", "--", file).RunInDirBytes(path)
+}
+
+// LineBlame returns the latest commit at the given line
+func (repo *Repository) LineBlame(revision, path, file string, line uint) (*Commit, error) {
+	res, err := NewCommand("blame", fmt.Sprintf("-L %d,%d", line, line), "-p", revision, "--", file).RunInDir(path)
+	if err != nil {
+		return nil, err
+	}
+	if len(res) < 40 {
+		return nil, fmt.Errorf("invalid result of blame: %s", res)
+	}
+	return repo.GetCommit(string(res[:40]))
 }

vendor/code.gitea.io/git/repo_commit.go

@@ -9,6 +9,8 @@ import (
 	"container/list"
 	"strconv"
 	"strings"
+
+	"github.com/mcuadros/go-version"
 )
 
 // GetRefCommitID returns the last commit ID string of given reference (branch or tag).
@@ -274,7 +276,7 @@ func (repo *Repository) CommitsCountBetween(start, end string) (int64, error) {
 func (repo *Repository) commitsBefore(id SHA1, limit int) (*list.List, error) {
 	cmd := NewCommand("log")
 	if limit > 0 {
-		cmd.AddArguments("-"+ strconv.Itoa(limit), prettyLogFormat, id.String())
+		cmd.AddArguments("-"+strconv.Itoa(limit), prettyLogFormat, id.String())
 	} else {
 		cmd.AddArguments(prettyLogFormat, id.String())
 	}
@@ -316,15 +318,35 @@ func (repo *Repository) getCommitsBeforeLimit(id SHA1, num int) (*list.List, err
 }
 
 func (repo *Repository) getBranches(commit *Commit, limit int) ([]string, error) {
-	stdout, err := NewCommand("for-each-ref", "--count="+ strconv.Itoa(limit), "--format=%(refname)", "--contains", commit.ID.String(), BranchPrefix).RunInDir(repo.Path)
+	if version.Compare(gitVersion, "2.7.0", ">=") {
+		stdout, err := NewCommand("for-each-ref", "--count="+strconv.Itoa(limit), "--format=%(refname:strip=2)", "--contains", commit.ID.String(), BranchPrefix).RunInDir(repo.Path)
+		if err != nil {
+			return nil, err
+		}
+
+		branches := strings.Fields(stdout)
+		return branches, nil
+	}
+
+	stdout, err := NewCommand("branch", "--contains", commit.ID.String()).RunInDir(repo.Path)
 	if err != nil {
 		return nil, err
 	}
 
 	refs := strings.Split(stdout, "\n")
-	branches := make([]string, len(refs)-1)
-	for i, ref := range refs[:len(refs)-1] {
-		branches[i] = strings.TrimPrefix(ref, BranchPrefix)
+
+	var max int
+	if len(refs) > limit {
+		max = limit
+	} else {
+		max = len(refs) - 1
+	}
+
+	branches := make([]string, max)
+	for i, ref := range refs[:max] {
+		parts := strings.Fields(ref)
+
+		branches[i] = parts[len(parts)-1]
 	}
 	return branches, nil
 }

vendor/code.gitea.io/git/sha1.go

@@ -5,6 +5,7 @@
 package git
 
 import (
+	"bytes"
 	"encoding/hex"
 	"fmt"
 	"strings"
@@ -26,43 +27,23 @@ func (id SHA1) Equal(s2 interface{}) bool {
 		}
 		return v == id.String()
 	case []byte:
-		if len(v) != 20 {
-			return false
-		}
-		for i, v := range v {
-			if id[i] != v {
-				return false
-			}
-		}
+		return bytes.Equal(v, id[:])
 	case SHA1:
-		for i, v := range v {
-			if id[i] != v {
-				return false
-			}
-		}
+		return v == id
 	default:
 		return false
 	}
-	return true
 }
 
 // String returns string (hex) representation of the Oid.
 func (id SHA1) String() string {
-	result := make([]byte, 0, 40)
-	hexvalues := []byte("0123456789abcdef")
-	for i := 0; i < 20; i++ {
-		result = append(result, hexvalues[id[i]>>4])
-		result = append(result, hexvalues[id[i]&0xf])
-	}
-	return string(result)
+	return hex.EncodeToString(id[:])
 }
 
 // MustID always creates a new SHA1 from a [20]byte array with no validation of input.
 func MustID(b []byte) SHA1 {
 	var id SHA1
-	for i := 0; i < 20; i++ {
-		id[i] = b[i]
-	}
+	copy(id[:], b)
 	return id
 }
 

vendor/code.gitea.io/git/tree.go

@@ -5,8 +5,6 @@
 package git
 
 import (
-	"bytes"
-	"fmt"
 	"strings"
 )
 
@@ -30,84 +28,6 @@ func NewTree(repo *Repository, id SHA1) *Tree {
 	}
 }
 
-var escapeChar = []byte("\\")
-
-// UnescapeChars reverses escaped characters.
-func UnescapeChars(in []byte) []byte {
-	if bytes.Index(in, escapeChar) == -1 {
-		return in
-	}
-
-	endIdx := len(in) - 1
-	isEscape := false
-	out := make([]byte, 0, endIdx+1)
-	for i := range in {
-		if in[i] == '\\' && !isEscape {
-			isEscape = true
-			continue
-		}
-		isEscape = false
-		out = append(out, in[i])
-	}
-	return out
-}
-
-// parseTreeData parses tree information from the (uncompressed) raw
-// data from the tree object.
-func parseTreeData(tree *Tree, data []byte) ([]*TreeEntry, error) {
-	entries := make([]*TreeEntry, 0, 10)
-	l := len(data)
-	pos := 0
-	for pos < l {
-		entry := new(TreeEntry)
-		entry.ptree = tree
-		step := 6
-		switch string(data[pos : pos+step]) {
-		case "100644":
-			entry.mode = EntryModeBlob
-			entry.Type = ObjectBlob
-		case "100755":
-			entry.mode = EntryModeExec
-			entry.Type = ObjectBlob
-		case "120000":
-			entry.mode = EntryModeSymlink
-			entry.Type = ObjectBlob
-		case "160000":
-			entry.mode = EntryModeCommit
-			entry.Type = ObjectCommit
-
-			step = 8
-		case "040000":
-			entry.mode = EntryModeTree
-			entry.Type = ObjectTree
-		default:
-			return nil, fmt.Errorf("unknown type: %v", string(data[pos:pos+step]))
-		}
-		pos += step + 6 // Skip string type of entry type.
-
-		step = 40
-		id, err := NewIDFromString(string(data[pos : pos+step]))
-		if err != nil {
-			return nil, err
-		}
-		entry.ID = id
-		pos += step + 1 // Skip half of SHA1.
-
-		step = bytes.IndexByte(data[pos:], '\n')
-
-		// In case entry name is surrounded by double quotes(it happens only in git-shell).
-		if data[pos] == '"' {
-			entry.name = string(UnescapeChars(data[pos+1 : pos+step-1]))
-		} else {
-			entry.name = string(data[pos : pos+step])
-		}
-
-		pos += step + 1
-		entries = append(entries, entry)
-	}
-	return entries, nil
-}
-
 // SubTree get a sub tree by the sub dir path
 func (t *Tree) SubTree(rpath string) (*Tree, error) {
 	if len(rpath) == 0 {
@@ -142,12 +62,11 @@ func (t *Tree) ListEntries() (Entries, error) {
 	if t.entriesParsed {
 		return t.entries, nil
 	}
-	t.entriesParsed = true
 
 	stdout, err := NewCommand("ls-tree", t.ID.String()).RunInDirBytes(t.repo.Path)
 	if err != nil {
 		return nil, err
 	}
-	t.entries, err = parseTreeData(t, stdout)
+	t.entries, err = parseTreeEntries(stdout, t)
 	return t.entries, err
 }

vendor/code.gitea.io/git/tree_entry.go

@@ -5,6 +5,7 @@
 package git
 
 import (
+	"io"
 	"sort"
 	"strconv"
 	"strings"
@@ -90,6 +91,45 @@ func (te *TreeEntry) Blob() *Blob {
 	}
 }
 
+// FollowLink returns the entry pointed to by a symlink
+func (te *TreeEntry) FollowLink() (*TreeEntry, error) {
+	if !te.IsLink() {
+		return nil, ErrBadLink{te.Name(), "not a symlink"}
+	}
+
+	// read the link
+	r, err := te.Blob().Data()
+	if err != nil {
+		return nil, err
+	}
+	buf := make([]byte, te.Size())
+	_, err = io.ReadFull(r, buf)
+	if err != nil {
+		return nil, err
+	}
+
+	lnk := string(buf)
+	t := te.ptree
+
+	// traverse up directories
+	for ; t != nil && strings.HasPrefix(lnk, "../"); lnk = lnk[3:] {
+		t = t.ptree
+	}
+
+	if t == nil {
+		return nil, ErrBadLink{te.Name(), "points outside of repo"}
+	}
+
+	target, err := t.GetTreeEntryByPath(lnk)
+	if err != nil {
+		if IsErrNotExist(err) {
+			return nil, ErrBadLink{te.Name(), "broken link"}
+		}
+		return nil, err
+	}
+	return target, nil
+}
+
 // GetSubJumpablePathName return the full path of subdirectory jumpable ( contains only one directory )
 func (te *TreeEntry) GetSubJumpablePathName() string {
 	if te.IsSubModule() || !te.IsDir() {

vendor/github.com/markbates/goth/README.md

@@ -8,6 +8,10 @@ protocol providers, as long as they implement the `Provider` and `Session` inter
 
 This package was inspired by [https://github.com/intridea/omniauth](https://github.com/intridea/omniauth).
 
+## Goth Needs a New Maintainer
+
+[https://blog.gobuffalo.io/goth-needs-a-new-maintainer-626cd47ca37b](https://blog.gobuffalo.io/goth-needs-a-new-maintainer-626cd47ca37b) - TL;DR: I, @markbates, won't be responding to any more issues, PRs, etc... for this package. A new maintainer needs to be found ASAP. Is this you?
+
 ## Installation
 
 ```text
@@ -18,6 +22,8 @@ $ go get github.com/markbates/goth
 
 * Amazon
 * Auth0
+* Azure AD
+* Battle.net
 * Bitbucket
 * Box
 * Cloud Foundry
@@ -26,6 +32,7 @@ $ go get github.com/markbates/goth
 * Digital Ocean
 * Discord
 * Dropbox
+* Eve Online
 * Facebook
 * Fitbit
 * GitHub
@@ -38,6 +45,7 @@ $ go get github.com/markbates/goth
 * Lastfm
 * Linkedin
 * Meetup
+* MicrosoftOnline
 * OneDrive
 * OpenID Connect (auto discovery)
 * Paypal
@@ -50,7 +58,9 @@ $ go get github.com/markbates/goth
 * Twitch
 * Twitter
 * Uber
+* VK
 * Wepay
+* Xero
 * Yahoo
 * Yammer
 
@@ -71,17 +81,51 @@ $ go get github.com/markbates/goth
 ```text
 $ cd goth/examples
 $ go get -v
-$ go build 
+$ go build
 $ ./examples
 ```
 
 Now open up your browser and go to [http://localhost:3000](http://localhost:3000) to see the example.
 
-To actually use the different providers, please make sure you configure them given the system environments as defined in the examples/main.go file
+To actually use the different providers, please make sure you set environment variables. Example given in the examples/main.go file
+
+## Security Notes
+
+By default, gothic uses a `CookieStore` from the `gorilla/sessions` package to store session data.
+
+As configured, this default store (`gothic.Store`) will generate cookies with `Options`:
+
+```go
+&Options{
+   Path:   "/",
+   Domain: "",
+   MaxAge: 86400 * 30,
+   HttpOnly: true,
+   Secure: false,
+ }
+```
+
+To tailor these fields for your application, you can override the `gothic.Store` variable at startup.
+
+The follow snippet show one way to do this:
+
+```go
+key := ""             // Replace with your SESSION_SECRET or similar
+maxAge := 86400 * 30  // 30 days
+isProd := false       // Set to true when serving over https
+
+store := sessions.NewCookieStore([]byte(key))
+store.MaxAge(maxAge)
+store.Options.Path = "/"
+store.Options.HttpOnly = true   // HttpOnly should always be enabled
+store.Options.Secure = isProd
+
+gothic.Store = store
+```
 
 ## Issues
 
-Issues always stand a significantly better chance of getting fixed if the are accompanied by a
+Issues always stand a significantly better chance of getting fixed if they are accompanied by a
 pull request.
 
 ## Contributing
@@ -94,50 +138,3 @@ Would I love to see more providers? Certainly! Would you love to contribute one?
 4. Commit your changes (git commit -am 'Add some feature')
 5. Push to the branch (git push origin my-new-feature)
 6. Create new Pull Request
-
-## Contributors
-
-* Mark Bates
-* Tyler Bunnell
-* Corey McGrillis
-* willemvd
-* Rakesh Goyal
-* Andy Grunwald
-* Glenn Walker
-* Kevin Fitzpatrick
-* Ben Tranter
-* Sharad Ganapathy
-* Andrew Chilton
-* sharadgana
-* Aurorae
-* Craig P Jolicoeur
-* Zac Bergquist
-* Geoff Franks
-* Raphael Geronimi
-* Noah Shibley
-* lumost
-* oov
-* Felix Lamouroux
-* Rafael Quintela
-* Tyler
-* DenSm
-* Samy KACIMI
-* dante gray
-* Noah
-* Jacob Walker
-* Marin Martinic
-* Roy
-* Omni Adams
-* Sasa Brankovic
-* dkhamsing
-* Dante Swift
-* Attila Domokos
-* Albin Gilles
-* Syed Zubairuddin
-* Johnny Boursiquot
-* Jerome Touffe-Blin
-* bryanl
-* Masanobu YOSHIOKA
-* Jonathan Hall
-* HaiMing.Yin
-* Sairam Kunala

vendor/github.com/markbates/goth/gothic/gothic.go

@@ -8,10 +8,18 @@ See https://github.com/markbates/goth/examples/main.go to see this in action.
 package gothic
 
 import (
+	"bytes"
+	"compress/gzip"
+	"encoding/base64"
 	"errors"
 	"fmt"
+	"io/ioutil"
+	"math/rand"
 	"net/http"
+	"net/url"
 	"os"
+	"strings"
+	"time"
 
 	"github.com/gorilla/mux"
 	"github.com/gorilla/sessions"
@@ -27,15 +35,21 @@ var defaultStore sessions.Store
 
 var keySet = false
 
+var gothicRand *rand.Rand
+
 func init() {
 	key := []byte(os.Getenv("SESSION_SECRET"))
 	keySet = len(key) != 0
-	Store = sessions.NewCookieStore([]byte(key))
+
+	cookieStore := sessions.NewCookieStore([]byte(key))
+	cookieStore.Options.HttpOnly = true
+	Store = cookieStore
 	defaultStore = Store
+	gothicRand = rand.New(rand.NewSource(time.Now().UnixNano()))
 }
 
 /*
-BeginAuthHandler is a convienence handler for starting the authentication process.
+BeginAuthHandler is a convenience handler for starting the authentication process.
 It expects to be able to get the name of the provider from the query parameters
 as either "provider" or ":provider".
 
@@ -65,8 +79,16 @@ var SetState = func(req *http.Request) string {
 		return state
 	}
 
-	return "state"
-
+	// If a state query param is not passed in, generate a random
+	// base64-encoded nonce so that the state on the auth URL
+	// is unguessable, preventing CSRF attacks, as described in
+	//
+	// https://auth0.com/docs/protocols/oauth2/oauth-state#keep-reading
+	nonceBytes := make([]byte, 64)
+	for i := 0; i < 64; i++ {
+		nonceBytes[i] = byte(gothicRand.Int63() % 256)
+	}
+	return base64.URLEncoding.EncodeToString(nonceBytes)
 }
 
 // GetState gets the state returned by the provider during the callback.
@@ -87,7 +109,6 @@ I would recommend using the BeginAuthHandler instead of doing all of these steps
 yourself, but that's entirely up to you.
 */
 func GetAuthURL(res http.ResponseWriter, req *http.Request) (string, error) {
-
 	if !keySet && defaultStore == Store {
 		fmt.Println("goth/gothic: no SESSION_SECRET environment variable is set. The default cookie store is not available and any calls will fail. Ignore this warning if you are using a different store.")
 	}
@@ -111,7 +132,7 @@ func GetAuthURL(res http.ResponseWriter, req *http.Request) (string, error) {
 		return "", err
 	}
 
-	err = storeInSession(providerName, sess.Marshal(), req, res)
+	err = StoreInSession(providerName, sess.Marshal(), req, res)
 
 	if err != nil {
 		return "", err
@@ -130,7 +151,7 @@ as either "provider" or ":provider".
 See https://github.com/markbates/goth/examples/main.go to see this in action.
 */
 var CompleteUserAuth = func(res http.ResponseWriter, req *http.Request) (goth.User, error) {
-
+	defer Logout(res, req)
 	if !keySet && defaultStore == Store {
 		fmt.Println("goth/gothic: no SESSION_SECRET environment variable is set. The default cookie store is not available and any calls will fail. Ignore this warning if you are using a different store.")
 	}
@@ -145,7 +166,7 @@ var CompleteUserAuth = func(res http.ResponseWriter, req *http.Request) (goth.Us
 		return goth.User{}, err
 	}
 
-	value, err := getFromSession(providerName, req)
+	value, err := GetFromSession(providerName, req)
 	if err != nil {
 		return goth.User{}, err
 	}
@@ -155,6 +176,11 @@ var CompleteUserAuth = func(res http.ResponseWriter, req *http.Request) (goth.Us
 		return goth.User{}, err
 	}
 
+	err = validateState(req, sess)
+	if err != nil {
+		return goth.User{}, err
+	}
+
 	user, err := provider.FetchUser(sess)
 	if err == nil {
 		// user can be found with existing session data
@@ -167,13 +193,49 @@ var CompleteUserAuth = func(res http.ResponseWriter, req *http.Request) (goth.Us
 		return goth.User{}, err
 	}
 
-	err = storeInSession(providerName, sess.Marshal(), req, res)
+	err = StoreInSession(providerName, sess.Marshal(), req, res)
 
 	if err != nil {
 		return goth.User{}, err
 	}
 
-	return provider.FetchUser(sess)
+	gu, err := provider.FetchUser(sess)
+	return gu, err
+}
+
+// validateState ensures that the state token param from the original
+// AuthURL matches the one included in the current (callback) request.
+func validateState(req *http.Request, sess goth.Session) error {
+	rawAuthURL, err := sess.GetAuthURL()
+	if err != nil {
+		return err
+	}
+
+	authURL, err := url.Parse(rawAuthURL)
+	if err != nil {
+		return err
+	}
+
+	originalState := authURL.Query().Get("state")
+	if originalState != "" && (originalState != req.URL.Query().Get("state")) {
+		return errors.New("state token mismatch")
+	}
+	return nil
+}
+
+// Logout invalidates a user session.
+func Logout(res http.ResponseWriter, req *http.Request) error {
+	session, err := Store.Get(req, SessionName)
+	if err != nil {
+		return err
+	}
+	session.Options.MaxAge = -1
+	session.Values = make(map[interface{}]interface{})
+	err = session.Save(req, res)
+	if err != nil {
+		return errors.New("Could not delete user session ")
+	}
+	return nil
 }
 
 // GetProviderName is a function used to get the name of a provider
@@ -184,36 +246,99 @@ var CompleteUserAuth = func(res http.ResponseWriter, req *http.Request) (goth.Us
 var GetProviderName = getProviderName
 
 func getProviderName(req *http.Request) (string, error) {
-	provider := req.URL.Query().Get("provider")
-	if provider == "" {
-		if p, ok := mux.Vars(req)["provider"]; ok {
+
+	// get all the used providers
+	providers := goth.GetProviders()
+
+	// loop over the used providers, if we already have a valid session for any provider (ie. user is already logged-in with a provider), then return that provider name
+	for _, provider := range providers {
+		p := provider.Name()
+		session, _ := Store.Get(req, p+SessionName)
+		value := session.Values[p]
+		if _, ok := value.(string); ok {
 			return p, nil
 		}
 	}
-	if provider == "" {
-		provider = req.URL.Query().Get(":provider")
+
+	// try to get it from the url param "provider"
+	if p := req.URL.Query().Get("provider"); p != "" {
+		return p, nil
 	}
-	if provider == "" {
-		return provider, errors.New("you must select a provider")
+
+	// try to get it from the url param ":provider"
+	if p := req.URL.Query().Get(":provider"); p != "" {
+		return p, nil
 	}
-	return provider, nil
+
+	// try to get it from the context's value of "provider" key
+	if p, ok := mux.Vars(req)["provider"]; ok {
+		return p, nil
+	}
+
+	//  try to get it from the go-context's value of "provider" key
+	if p, ok := req.Context().Value("provider").(string); ok {
+		return p, nil
+	}
+
+	// if not found then return an empty string with the corresponding error
+	return "", errors.New("you must select a provider")
 }
 
-func storeInSession(key string, value string, req *http.Request, res http.ResponseWriter) error {
-	session, _ := Store.Get(req, key + SessionName)
+// StoreInSession stores a specified key/value pair in the session.
+func StoreInSession(key string, value string, req *http.Request, res http.ResponseWriter) error {
+	session, _ := Store.New(req, SessionName)
 
-	session.Values[key] = value
+	if err := updateSessionValue(session, key, value); err != nil {
+		return err
+	}
 
 	return session.Save(req, res)
 }
 
-func getFromSession(key string, req *http.Request) (string, error) {
-	session, _ := Store.Get(req, key + SessionName)
-
-	value := session.Values[key]
-	if value == nil {
+// GetFromSession retrieves a previously-stored value from the session.
+// If no value has previously been stored at the specified key, it will return an error.
+func GetFromSession(key string, req *http.Request) (string, error) {
+	session, _ := Store.Get(req, SessionName)
+	value, err := getSessionValue(session, key)
+	if err != nil {
 		return "", errors.New("could not find a matching session for this request")
 	}
 
-	return value.(string), nil
-}
+	return value, nil
+}
+
+func getSessionValue(session *sessions.Session, key string) (string, error) {
+	value := session.Values[key]
+	if value == nil {
+		return "", fmt.Errorf("could not find a matching session for this request")
+	}
+
+	rdata := strings.NewReader(value.(string))
+	r, err := gzip.NewReader(rdata)
+	if err != nil {
+		return "", err
+	}
+	s, err := ioutil.ReadAll(r)
+	if err != nil {
+		return "", err
+	}
+
+	return string(s), nil
+}
+
+func updateSessionValue(session *sessions.Session, key, value string) error {
+	var b bytes.Buffer
+	gz := gzip.NewWriter(&b)
+	if _, err := gz.Write([]byte(value)); err != nil {
+		return err
+	}
+	if err := gz.Flush(); err != nil {
+		return err
+	}
+	if err := gz.Close(); err != nil {
+		return err
+	}
+
+	session.Values[key] = b.String()
+	return nil
+}

vendor/github.com/markbates/goth/providers/bitbucket/bitbucket.go

@@ -9,9 +9,9 @@ import (
 	"net/http"
 	"net/url"
 
+	"fmt"
 	"github.com/markbates/goth"
 	"golang.org/x/oauth2"
-	"fmt"
 )
 
 const (
@@ -26,10 +26,10 @@ const (
 // one manually.
 func New(clientKey, secret, callbackURL string, scopes ...string) *Provider {
 	p := &Provider{
-		ClientKey:           clientKey,
-		Secret:              secret,
-		CallbackURL:         callbackURL,
-		providerName:        "bitbucket",
+		ClientKey:    clientKey,
+		Secret:       secret,
+		CallbackURL:  callbackURL,
+		providerName: "bitbucket",
 	}
 	p.config = newConfig(p, scopes)
 	return p
@@ -125,7 +125,7 @@ func (p *Provider) FetchUser(session goth.Session) (goth.User, error) {
 
 func userFromReader(reader io.Reader, user *goth.User) error {
 	u := struct {
-		ID string `json:"uuid"`
+		ID    string `json:"uuid"`
 		Links struct {
 			Avatar struct {
 				URL string `json:"href"`

vendor/github.com/markbates/goth/providers/dropbox/dropbox.go

@@ -2,21 +2,24 @@
 package dropbox
 
 import (
+	"bytes"
 	"encoding/json"
 	"errors"
 	"io"
+	"io/ioutil"
 	"net/http"
 	"strings"
 
+	"fmt"
+
 	"github.com/markbates/goth"
 	"golang.org/x/oauth2"
-	"fmt"
 )
 
 const (
-	authURL    = "https://www.dropbox.com/1/oauth2/authorize"
-	tokenURL   = "https://api.dropbox.com/1/oauth2/token"
-	accountURL = "https://api.dropbox.com/1/account/info"
+	authURL    = "https://www.dropbox.com/oauth2/authorize"
+	tokenURL   = "https://api.dropbox.com/oauth2/token"
+	accountURL = "https://api.dropbox.com/2/users/get_current_account"
 )
 
 // Provider is the implementation of `goth.Provider` for accessing Dropbox.
@@ -24,6 +27,7 @@ type Provider struct {
 	ClientKey    string
 	Secret       string
 	CallbackURL  string
+	AccountURL   string
 	HTTPClient   *http.Client
 	config       *oauth2.Config
 	providerName string
@@ -40,10 +44,11 @@ type Session struct {
 // create one manually.
 func New(clientKey, secret, callbackURL string, scopes ...string) *Provider {
 	p := &Provider{
-		ClientKey:           clientKey,
-		Secret:              secret,
-		CallbackURL:         callbackURL,
-		providerName:        "dropbox",
+		ClientKey:    clientKey,
+		Secret:       secret,
+		CallbackURL:  callbackURL,
+		AccountURL:   accountURL,
+		providerName: "dropbox",
 	}
 	p.config = newConfig(p, scopes)
 	return p
@@ -86,7 +91,7 @@ func (p *Provider) FetchUser(session goth.Session) (goth.User, error) {
 		return user, fmt.Errorf("%s cannot get user information without accessToken", p.providerName)
 	}
 
-	req, err := http.NewRequest("GET", accountURL, nil)
+	req, err := http.NewRequest("POST", p.AccountURL, nil)
 	if err != nil {
 		return user, err
 	}
@@ -101,7 +106,17 @@ func (p *Provider) FetchUser(session goth.Session) (goth.User, error) {
 		return user, fmt.Errorf("%s responded with a %d trying to fetch user information", p.providerName, resp.StatusCode)
 	}
 
-	err = userFromReader(resp.Body, &user)
+	bits, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return user, err
+	}
+
+	err = json.NewDecoder(bytes.NewReader(bits)).Decode(&user.RawData)
+	if err != nil {
+		return user, err
+	}
+
+	err = userFromReader(bytes.NewReader(bits), &user)
 	return user, err
 }
 
@@ -161,22 +176,29 @@ func newConfig(p *Provider, scopes []string) *oauth2.Config {
 
 func userFromReader(r io.Reader, user *goth.User) error {
 	u := struct {
-		Name string `json:"display_name"`
-		NameDetails struct {
-			NickName string `json:"familiar_name"`
-		} `json:"name_details"`
-		Location string `json:"country"`
-		Email    string `json:"email"`
+		AccountID string `json:"account_id"`
+		Name      struct {
+			GivenName   string `json:"given_name"`
+			Surname     string `json:"surname"`
+			DisplayName string `json:"display_name"`
+		} `json:"name"`
+		Country         string `json:"country"`
+		Email           string `json:"email"`
+		ProfilePhotoURL string `json:"profile_photo_url"`
 	}{}
 	err := json.NewDecoder(r).Decode(&u)
 	if err != nil {
 		return err
 	}
+	user.UserID = u.AccountID // The user's unique Dropbox ID.
+	user.FirstName = u.Name.GivenName
+	user.LastName = u.Name.Surname
+	user.Name = strings.TrimSpace(fmt.Sprintf("%s %s", u.Name.GivenName, u.Name.Surname))
+	user.Description = u.Name.DisplayName // Full name plus parenthetical team naem
 	user.Email = u.Email
-	user.Name = u.Name
-	user.NickName = u.NameDetails.NickName
-	user.UserID = u.Email // Dropbox doesn't provide a separate user ID
-	user.Location = u.Location
+	user.NickName = u.Email // Email is the dropbox username
+	user.Location = u.Country
+	user.AvatarURL = u.ProfilePhotoURL // May be blank
 	return nil
 }
 

vendor/github.com/markbates/goth/providers/facebook/facebook.go

@@ -11,12 +11,12 @@ import (
 	"net/http"
 	"net/url"
 
-	"github.com/markbates/goth"
-	"golang.org/x/oauth2"
-	"fmt"
 	"crypto/hmac"
 	"crypto/sha256"
 	"encoding/hex"
+	"fmt"
+	"github.com/markbates/goth"
+	"golang.org/x/oauth2"
 )
 
 const (
@@ -30,10 +30,10 @@ const (
 // one manually.
 func New(clientKey, secret, callbackURL string, scopes ...string) *Provider {
 	p := &Provider{
-		ClientKey:           clientKey,
-		Secret:              secret,
-		CallbackURL:         callbackURL,
-		providerName:        "facebook",
+		ClientKey:    clientKey,
+		Secret:       secret,
+		CallbackURL:  callbackURL,
+		providerName: "facebook",
 	}
 	p.config = newConfig(p, scopes)
 	return p
@@ -129,7 +129,7 @@ func userFromReader(reader io.Reader, user *goth.User) error {
 		FirstName string `json:"first_name"`
 		LastName  string `json:"last_name"`
 		Link      string `json:"link"`
-		Picture struct {
+		Picture   struct {
 			Data struct {
 				URL string `json:"url"`
 			} `json:"data"`

vendor/github.com/markbates/goth/providers/gitlab/gitlab.go

@@ -11,9 +11,9 @@ import (
 	"net/url"
 	"strconv"
 
+	"fmt"
 	"github.com/markbates/goth"
 	"golang.org/x/oauth2"
-	"fmt"
 )
 
 // These vars define the Authentication, Token, and Profile URLS for Gitlab. If

vendor/github.com/markbates/goth/providers/gplus/gplus.go

@@ -11,9 +11,9 @@ import (
 	"net/url"
 	"strings"
 
+	"fmt"
 	"github.com/markbates/goth"
 	"golang.org/x/oauth2"
-	"fmt"
 )
 
 const (
@@ -27,10 +27,10 @@ const (
 // one manually.
 func New(clientKey, secret, callbackURL string, scopes ...string) *Provider {
 	p := &Provider{
-		ClientKey:           clientKey,
-		Secret:              secret,
-		CallbackURL:         callbackURL,
-		providerName:        "gplus",
+		ClientKey:    clientKey,
+		Secret:       secret,
+		CallbackURL:  callbackURL,
+		providerName: "gplus",
 	}
 	p.config = newConfig(p, scopes)
 	return p

vendor/github.com/markbates/goth/providers/openidConnect/openidConnect.go

@@ -1,17 +1,17 @@
 package openidConnect
 
 import (
+	"bytes"
+	"encoding/base64"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"github.com/markbates/goth"
+	"golang.org/x/oauth2"
+	"io/ioutil"
 	"net/http"
 	"strings"
-	"fmt"
-	"encoding/json"
-	"encoding/base64"
-	"io/ioutil"
-	"errors"
-	"golang.org/x/oauth2"
-	"github.com/markbates/goth"
 	"time"
-	"bytes"
 )
 
 const (
@@ -89,14 +89,14 @@ func New(clientKey, secret, callbackURL, openIDAutoDiscoveryURL string, scopes .
 		Secret:      secret,
 		CallbackURL: callbackURL,
 
-		UserIdClaims:   []string{subjectClaim},
-		NameClaims:     []string{NameClaim},
-		NickNameClaims: []string{NicknameClaim, PreferredUsernameClaim},
-		EmailClaims:    []string{EmailClaim},
-		AvatarURLClaims:[]string{PictureClaim},
-		FirstNameClaims:[]string{GivenNameClaim},
-		LastNameClaims: []string{FamilyNameClaim},
-		LocationClaims: []string{AddressClaim},
+		UserIdClaims:    []string{subjectClaim},
+		NameClaims:      []string{NameClaim},
+		NickNameClaims:  []string{NicknameClaim, PreferredUsernameClaim},
+		EmailClaims:     []string{EmailClaim},
+		AvatarURLClaims: []string{PictureClaim},
+		FirstNameClaims: []string{GivenNameClaim},
+		LastNameClaims:  []string{FamilyNameClaim},
+		LocationClaims:  []string{AddressClaim},
 
 		providerName: "openid-connect",
 	}

vendor/github.com/markbates/goth/providers/openidConnect/session.go

@@ -1,12 +1,12 @@
 package openidConnect
 
 import (
+	"encoding/json"
 	"errors"
 	"github.com/markbates/goth"
-	"encoding/json"
+	"golang.org/x/oauth2"
 	"strings"
 	"time"
-	"golang.org/x/oauth2"
 )
 
 // Session stores data during the auth process with the OpenID Connect provider.

vendor/github.com/markbates/goth/providers/twitter/twitter.go

@@ -9,10 +9,11 @@ import (
 	"io/ioutil"
 	"net/http"
 
+	"fmt"
+
 	"github.com/markbates/goth"
 	"github.com/mrjones/oauth"
 	"golang.org/x/oauth2"
-	"fmt"
 )
 
 var (
@@ -30,10 +31,10 @@ var (
 // If you'd like to use authenticate instead of authorize, use NewAuthenticate instead.
 func New(clientKey, secret, callbackURL string) *Provider {
 	p := &Provider{
-		ClientKey:           clientKey,
-		Secret:              secret,
-		CallbackURL:         callbackURL,
-		providerName:        "twitter",
+		ClientKey:    clientKey,
+		Secret:       secret,
+		CallbackURL:  callbackURL,
+		providerName: "twitter",
 	}
 	p.consumer = newConsumer(p, authorizeURL)
 	return p
@@ -43,10 +44,10 @@ func New(clientKey, secret, callbackURL string) *Provider {
 // NewAuthenticate uses the authenticate URL instead of the authorize URL.
 func NewAuthenticate(clientKey, secret, callbackURL string) *Provider {
 	p := &Provider{
-		ClientKey:           clientKey,
-		Secret:              secret,
-		CallbackURL:         callbackURL,
-		providerName:        "twitter",
+		ClientKey:    clientKey,
+		Secret:       secret,
+		CallbackURL:  callbackURL,
+		providerName: "twitter",
 	}
 	p.consumer = newConsumer(p, authenticateURL)
 	return p
@@ -107,7 +108,7 @@ func (p *Provider) FetchUser(session goth.Session) (goth.User, error) {
 
 	response, err := p.consumer.Get(
 		endpointProfile,
-		map[string]string{"include_entities": "false", "skip_status": "true"},
+		map[string]string{"include_entities": "false", "skip_status": "true", "include_email": "true"},
 		sess.AccessToken)
 	if err != nil {
 		return user, err
@@ -126,6 +127,9 @@ func (p *Provider) FetchUser(session goth.Session) (goth.User, error) {
 
 	user.Name = user.RawData["name"].(string)
 	user.NickName = user.RawData["screen_name"].(string)
+	if user.RawData["email"] != nil {
+		user.Email = user.RawData["email"].(string)
+	}
 	user.Description = user.RawData["description"].(string)
 	user.AvatarURL = user.RawData["profile_image_url"].(string)
 	user.UserID = user.RawData["id_str"].(string)

vendor/vendor.json

@@ -3,10 +3,10 @@
 	"ignore": "test appengine",
 	"package": [
 		{
-			"checksumSHA1": "Gz+a5Qo4PCiB/Gf2f02v8HEAxDM=",
+			"checksumSHA1": "xwQNnA5geMAdbiBjBABtsjqZZMw=",
 			"path": "code.gitea.io/git",
-			"revision": "6798d0f202cdc7187c00a467b586a4bdee27e8c9",
-			"revisionTime": "2018-01-14T14:37:32Z"
+			"revision": "31f4b8e8c805438ac6d8914b38accb1d8aaf695e",
+			"revisionTime": "2018-05-26T05:17:21Z"
 		},
 		{
 			"checksumSHA1": "Qtq0kW+BnpYMOriaoCjMa86WGG8=",
@@ -654,64 +654,74 @@
 			"revisionTime": "2017-10-25T03:15:54Z"
 		},
 		{
-			"checksumSHA1": "O3KUfEXQPfdQ+tCMpP2RAIRJJqY=",
+			"checksumSHA1": "q9MD1ienC+kmKq5i51oAktQEV1E=",
+			"origin": "github.com/go-gitea/goth",
 			"path": "github.com/markbates/goth",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
-			"checksumSHA1": "MkFKwLV3icyUo4oP0BgEs+7+R1Y=",
+			"checksumSHA1": "FISfgOkoMtn98wglLUvfBTZ6baE=",
+			"origin": "github.com/go-gitea/goth/gothic",
 			"path": "github.com/markbates/goth/gothic",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
-			"checksumSHA1": "crNSlQADjX6hcxykON2tFCqY4iw=",
+			"checksumSHA1": "pJ+Cws/TU22K6tZ/ALFOvvH1K5U=",
+			"origin": "github.com/go-gitea/goth/providers/bitbucket",
 			"path": "github.com/markbates/goth/providers/bitbucket",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
-			"checksumSHA1": "1Kp4DKkJNVn135Xg8H4a6CFBNy8=",
+			"checksumSHA1": "XsF5HI4240QHbFXbtWWnGgTsoq8=",
+			"origin": "github.com/go-gitea/goth/providers/dropbox",
 			"path": "github.com/markbates/goth/providers/dropbox",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
-			"checksumSHA1": "cGs1da29iOBJh5EAH0icKDbN8CA=",
+			"checksumSHA1": "VzbroIA9R00Ig3iGnOlZLU7d4ls=",
+			"origin": "github.com/go-gitea/goth/providers/facebook",
 			"path": "github.com/markbates/goth/providers/facebook",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
 			"checksumSHA1": "P6nBZ850aaekpOcoXNdRhK86bH8=",
+			"origin": "github.com/go-gitea/goth/providers/github",
 			"path": "github.com/markbates/goth/providers/github",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
-			"checksumSHA1": "o/109paSRy9HqV87gR4zUZMMSzs=",
+			"checksumSHA1": "ld488t+yGoTwtmiCSSggEX4fxVk=",
+			"origin": "github.com/go-gitea/goth/providers/gitlab",
 			"path": "github.com/markbates/goth/providers/gitlab",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
-			"checksumSHA1": "cX6kR9y94BWFZvI/7UFrsFsP3FQ=",
+			"checksumSHA1": "qXEulD7vnwY9hFrxh91Pm5YrvTM=",
+			"origin": "github.com/go-gitea/goth/providers/gplus",
 			"path": "github.com/markbates/goth/providers/gplus",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
-			"checksumSHA1": "sMYKhqAUZXM1+T/TjlMhWh8Vveo=",
+			"checksumSHA1": "wsOBzyp4LKDhfCPmX1LLP7T0S3U=",
+			"origin": "github.com/go-gitea/goth/providers/openidConnect",
 			"path": "github.com/markbates/goth/providers/openidConnect",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
-			"checksumSHA1": "1w0V6jYXaGlEtZcMeYTOAAucvgw=",
+			"checksumSHA1": "o6RqMbbE8QNZhNT9TsAIRMPI8tg=",
+			"origin": "github.com/go-gitea/goth/providers/twitter",
 			"path": "github.com/markbates/goth/providers/twitter",
-			"revision": "90362394a367f9d77730911973462a53d69662ba",
-			"revisionTime": "2017-02-23T14:12:10Z"
+			"revision": "3b54d96084a5e11030f19556cf68a6ab5d93ba20",
+			"revisionTime": "2018-03-12T06:32:04Z"
 		},
 		{
 			"checksumSHA1": "61HNjGetaBoMp8HBOpuEZRSim8g=",