Commit Graph

39 Commits (4f82a0b12e1218c89e3300f3090c1a394fa33544)

Author SHA1 Message Date
Lunny Xiao e4e85a3e51
Storage configuration support `[storage]` (#13314)
* Fix minio bug

* Add tests for storage configuration

* Change the Seek flag to keep compitable minio?

* Fix test when first-byte-pos of all ranges is greater than the resource length

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-10-31 16:51:48 -04:00
Lunny Xiao 7a5465fc56
LFS support to be stored on minio (#12518)
* LFS support to be stored on minio

* Fix test

* Fix lint

* Fix lint

* Fix check

* Fix test

* Update documents and add migration for LFS

* Fix some bugs
2020-09-08 23:45:10 +08:00
zeripath 2ef318e6f1
Add Access-Control-Expose-Headers (#12446)
Fix #12424

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
2020-08-13 18:18:18 +01:00
zeripath 74bd9691c6
Re-attempt to delete temporary upload if the file is locked by another process (#12447)
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.

Fix #12339

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
2020-08-11 21:05:34 +01:00
burbon d8e6acda8c
Support Range header end in lfs (#11314)
* Initial support of end Range header option in lfs

* Allow end range option to be unspecified

* Declare toByte for consistency

* Factor out content encoding tests from doLfs

This is so Range tests could still use doLfs but without repeating
not related tests

* Add Range header test

* implemented extraHeader
* parametrized expectedStatus

* Add more test cases of Range header

* Fix S1030: should use resp.Body.String()

* Add more tests for edge cases

* Fix tests

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2020-05-11 11:37:59 +03:00
John Olheiser baadb51445
Add gitea-vet (#10948)
* Add copyright

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Add gitea-vet and fix non-compliance

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Combine tools.go into build.go and clean up

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Remove extra GO111MODULE=on

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2020-04-05 07:20:50 +01:00
zeripath 9269b7f627
Multiple LFS improvements (#10667)
* Add more logging in the LFS server

Adds more logging in the LFS server and stops sending internal server
error information to the client

* Add LFS Lock cursor implementation

* Simplify Claims in LFS and remove the float64 casts

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-03-09 19:56:18 +00:00
zeripath 0b3a9c4f85
Fix LFS max file size enforcement (#10593)
Unfortunately the batch handler also needs to enforce max file size.
Enforce max file size in batch handler
2020-03-03 14:57:27 -06:00
zeripath 513b962c1d
Add max-file-size to LFS (#10463)
* Add max-file-size to LFS

* Update modules/lfs/server.go

* As per @silverwind

Co-Authored-By: silverwind <me@silverwind.io>

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-02-28 01:46:57 -03:00
zeripath dc2fe9801f Make repository management section handle lfs locks (#8726)
* Make repository maangement section handle lfs locks

* Add check attribute handling and handle locking paths better

* More cleanly check-attributes

* handle error

* Check if file exists in default branch before linking to it.

* fixup

* Properly cleanPath

* Use cleanPath

* Sigh
2019-12-12 21:18:07 +08:00
zeripath 5e6a008fba
Add basic repository lfs management (#7199)
This PR adds basic repository LFS management UI including the ability to find all possible pointers within the repository. Locks are not managed at present but would be addable through some simple additions.

* Add basic repository lfs management
* add auto-associate function
* Add functionality to find commits with this lfs file
* Add link to find commits on the lfs file view
* Adjust commit view to state the likely branch causing the commit
* Only read Oid from database
2019-10-28 18:31:55 +00:00
zeripath 6551a9d6ca
Ensure Request Body Readers are closed in LFS server (#8454) 2019-10-10 18:42:28 +01:00
Tamal Saha 171b359877 Use gitea forked macaron (#7933)
Signed-off-by: Tamal Saha <tamal@appscode.com>
2019-08-23 12:40:29 -04:00
Christian Muehlhaeuser 54d96c79b5 Removed unnecessary conversions (#7557)
No need to convert to the same type.
2019-07-23 19:50:39 +01:00
kolaente f9ec2f89f2 Add golangci (#6418) 2019-06-12 15:41:28 -04:00
zeripath 31557b1274
Fix LFS Locks over SSH (#6999)
* Fix LFS Locks over SSH
* Mark test as skipped
2019-05-28 11:32:41 +01:00
Marat Radchenko 844f9a4bd8 Fix /verify LFS handler expecting wrong content-type (#7015)
Fixes #6960

According to [spec][1], /verify requests must have `Accept: application/vnd.git-lfs+json`

Previous code works because `git-lfs` also [violates spec and doesn't send any Accept header at all][2]
For other clients that DO set `Accept: application/vnd.git-lfs+json`, addition of `Accept: application/vnd.git-lfs`
either forces them to violate the spec or is ignored, depending on order in what they create header list.

[1]: https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md#verification
[2]: https://github.com/git-lfs/git-lfs/issues/3662
2019-05-24 22:21:00 +01:00
Lunny Xiao 34eee25bd4
Move sdk structs to modules/structs (#6905)
* move sdk structs to moduels/structs

* fix tests

* fix fmt

* fix swagger

* fix vendor
2019-05-11 18:21:34 +08:00
zeripath 704da08fdc
Better logging (#6038) (#6095)
* Panic don't fatal on create new logger

Fixes #5854

Signed-off-by: Andrew Thornton <art27@cantab.net>

* partial broken

* Update the logging infrastrcture

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Reset the skip levels for Fatal and Error

Signed-off-by: Andrew Thornton <art27@cantab.net>

* broken ncsa

* More log.Error fixes

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove nal

* set log-levels to lowercase

* Make console_test test all levels

* switch to lowercased levels

* OK now working

* Fix vetting issues

* Fix lint

* Fix tests

* change default logging to match current gitea

* Improve log testing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* reset error skip levels to 0

* Update documentation and access logger configuration

* Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE

* Fix broken level caching

* Refactor the router log

* Add Router logger

* Add colorizing options

* Adjust router colors

* Only create logger if they will be used

* update app.ini.sample

* rename Attribute ColorAttribute

* Change from white to green for function

* Set fatal/error levels

* Restore initial trace logger

* Fix Trace arguments in modules/auth/auth.go

* Properly handle XORMLogger

* Improve admin/config page

* fix fmt

* Add auto-compression of old logs

* Update error log levels

* Remove the unnecessary skip argument from Error, Fatal and Critical

* Add stacktrace support

* Fix tests

* Remove x/sync from vendors?

* Add stderr option to console logger

* Use filepath.ToSlash to protect against Windows in tests

* Remove prefixed underscores from names in colors.go

* Remove not implemented database logger

This was removed from Gogs on 4 Mar 2016 but left in the configuration
since then.

* Ensure that log paths are relative to ROOT_PATH

* use path.Join

* rename jsonConfig to logConfig

* Rename "config" to "jsonConfig" to make it clearer

* Requested changes

* Requested changes: XormLogger

* Try to color the windows terminal

If successful default to colorizing the console logs

* fixup

* Colorize initially too

* update vendor

* Colorize logs on default and remove if this is not a colorizing logger

* Fix documentation

* fix test

* Use go-isatty to detect if on windows we are on msys or cygwin

* Fix spelling mistake

* Add missing vendors

* More changes

* Rationalise the ANSI writer protection

* Adjust colors on advice from @0x5c

* Make Flags a comma separated list

* Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING

* Ensure matching is done on the non-colored message - to simpify EXPRESSION
2019-04-02 08:48:31 +01:00
zeripath 2a03e96bce
Allow markdown files to read from the LFS (#5787)
This PR makes it possible for the markdown renderer to render images and media straight from the LFS.

Fix #5746

Signed-off-by: Andrew Thornton [art27@cantab.net](mailto:art27@cantab.net)
2019-02-12 15:09:43 +00:00
Lunny Xiao bbad9b6178 Fix bug when read public repo lfs file (#5912)
* fix bug when read public repo lfs file

* add comment on lfs permission check
2019-01-31 13:36:57 +00:00
zeripath 0f295ababa Only allow local login if password is non-empty (#5906) 2019-01-30 23:18:54 +02:00
Lunny Xiao eabbddcd98
Restrict permission check on repositories and fix some problems (#5314)
* fix units permission problems

* fix some bugs and merge LoadUnits to repoAssignment

* refactor permission struct and add some copyright heads

* remove unused codes

* fix routes units check

* improve permission check

* add unit tests for permission

* fix typo

* fix tests

* fix some routes

* fix api permission check

* improve permission check

* fix some permission check

* fix tests

* fix tests

* improve some permission check

* fix some permission check

* refactor AccessLevel

* fix bug

* fix tests

* fix tests

* fix tests

* fix AccessLevel

* rename CanAccess

* fix tests

* fix comment

* fix bug

* add missing unit for test repos

* fix bug

* rename some functions

* fix routes check
2018-11-28 19:26:14 +08:00
Lunny Xiao d1337299e2 add valid for lfs oid (#4461) 2018-07-19 11:39:19 -04:00
Lauris BH 181b3a8f09 Fix path cleanup in multiple places (#3871) 2018-05-01 09:46:04 +08:00
Antoine GIRARD 9e842c8a72 Fix SSH auth lfs locks (#3152)
* Fix SSH auth LFS locks

* Activate SSH/lock test

* Remove debug

* Follow @lunny recommendation for AfterLoad method
2018-01-27 18:48:15 +02:00
Antoine GIRARD ef78309b65 Improve LFS tests + fix lfs url refs + keep path upper/lowercase in db. (#3092)
* Add failing test

* Fix urls

* Improve url in tests

* improve testing

* Remove debug code

* Add deps

* LFS corner-case : Search on lower but store with case

* Temporary comment of blocking action

* fix hooks

* Use temporary repo for git client test

* Use userPassword in place of hard-coded password
2017-12-08 20:21:37 +08:00
Lunny Xiao 35cc5b0402
Remove GetRepositoryByRef and add GetRepositoryByOwnerAndName (#3043)
* remove GetRepositoryByRef and add GetRepositoryByOwnerAndName

* fix tests

* fix tests bug

* some improvements
2017-12-02 15:34:39 +08:00
Antoine GIRARD 4035ab05fa Fix git lfs path (#3016)
* add suffix .git

* Remove Content-Type on GET methods

* Add migrations to force creation before any further potential migration
2017-11-29 01:35:23 +02:00
Antoine GIRARD d99f4ab003 Git LFS lock api (#2938)
* Implement routes

* move to api/sdk and create model

* Implement add + list

* List return 200 empty list no 404

* Add verify lfs lock api

* Add delete and start implementing auth control

* Revert to code.gitea.io/sdk/gitea vendor

* Apply needed check for all lfs locks route

* Add simple tests

* fix lint

* Improve tests

* Add delete test + fix

* Add lfs ascii header

* Various fixes from review + remove useless code + add more corner case testing

* Remove repo link since only id is needed.

Save a little of memory and cpu time.

* Improve tests

* Use TEXT column format for path + test

* fix mispell

* Use NewRequestWithJSON for POST tests

* Clean path

* Improve DB format

* Revert uniquess repoid+path

* (Re)-setup uniqueness + max path length

* Fixed TEXT in place of VARCHAR

* Settle back to maximum VARCHAR(3072)

* Let place for repoid in key

* Let place for repoid in key

* Let place for repoid in key

* Revert back
2017-11-28 22:58:37 +02:00
Lauris BH ba2e0240c6
Add LFS object verification step after upload (#2868)
* Add LFS object verification step after upload

* Fix file verification condition and small refactor

* Fix URLs

* Remove newline and return status 422 on failed verification

* Better error hadling
2017-11-08 15:04:19 +02:00
Jacob Parry 6e2e7b325f Fix LFS not returning correct content length when requesting a range of bytes (#2864) 2017-11-07 06:33:04 +02:00
Lauris BH 82fc50f4ee Fix Git LFS object/repo link storage in database and small refactoring (#2803) 2017-10-30 20:11:56 +08:00
Ethan Koenig 4cb9394a97 Fix lint errors (#2547) 2017-09-19 11:08:30 +03:00
Ethan Koenig 021904e4e6 Fix typos in models/ and modules/ (#1248) 2017-03-15 08:52:01 +08:00
Ethan Koenig ec0ae5d50c Refactor and fix incorrect comment (#1247) 2017-03-15 08:51:46 +08:00
Bo-Yi Wu 25663b5816 refactor: Remove unnecessary type conversions (#772) 2017-01-27 23:03:32 +08:00
Bo-Yi Wu 6510e57758 fix gofmt error
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
2016-12-30 20:41:10 +01:00
Fabian Zaremba 2e7ccecfe6 Git LFS support v2 (#122)
* Import github.com/git-lfs/lfs-test-server as lfs module base

Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198

Removed:

Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go
.dockerignore .gitignore README.md

* Remove config, add JWT support from github.com/mgit-at/lfs-test-server

Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83

* Add LFS settings

* Add LFS meta object model

* Add LFS routes and initialization

* Import github.com/dgrijalva/jwt-go into vendor/

* Adapt LFS module: handlers, routing, meta store

* Move LFS routes to /user/repo/info/lfs/*

* Add request header checks to LFS BatchHandler / PostHandler

* Implement LFS basic authentication

* Rework JWT secret generation / load

* Implement LFS SSH token authentication with JWT

Specification: https://github.com/github/git-lfs/tree/master/docs/api

* Integrate LFS settings into install process

* Remove LFS objects when repository is deleted

Only removes objects from content store when deleted repo is the only
referencing repository

* Make LFS module stateless

Fixes bug where LFS would not work after installation without
restarting Gitea

* Change 500 'Internal Server Error' to 400 'Bad Request'

* Change sql query to xorm call

* Remove unneeded type from LFS module

* Change internal imports to code.gitea.io/gitea/

* Add Gitea authors copyright

* Change basic auth realm to "gitea-lfs"

* Add unique indexes to LFS model

* Use xorm count function in LFS check on repository delete

* Return io.ReadCloser from content store and close after usage

* Add LFS info to runWeb()

* Export LFS content store base path

* LFS file download from UI

* Work around git-lfs client issue with unauthenticated requests

Returning a dummy Authorization header for unauthenticated requests
lets git-lfs client skip asking for auth credentials
See: https://github.com/github/git-lfs/issues/1088

* Fix unauthenticated UI downloads from public repositories

* Authentication check order, Finish LFS file view logic

* Ignore LFS hooks if installed for current OS user

Fixes Gitea UI actions for repositories tracking LFS files.
Checks for minimum needed git version by parsing the semantic version
string.

* Hide LFS metafile diff from commit view, marking as binary

* Show LFS notice if file in commit view is tracked

* Add notbefore/nbf JWT claim

* Correct lint suggestions - comments for structs and functions

- Add comments to LFS model
- Function comment for GetRandomBytesAsBase64
- LFS server function comments and lint variable suggestion

* Move secret generation code out of conditional

Ensures no LFS code may run with an empty secret

* Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 09:16:37 +08:00