Skip to main content
Git internals & repository operations
Git
バージョン管理
アーキテクチャ設計
信頼性
セキュリティ

How Git Actually Works: Objects, Refs, the Index, and Packfiles

A ground-up explanation of Git's internals, faithful to the official documentation: the content-addressed object database, the 41-byte ref, reftable, the DIRC index and conflict stages 1/2/3, three-way merges under the ort strategy, packfile delta compression, and the SHA-256 and Git 3.0 breaking changes — with plumbing-command output you can reproduce yourself.

Published
Reading time
27 min read
Author
友田 陽大
Share
Contents

You ran git reset --hard and three hours of work vanished. You ran git push --force and wiped out a teammate's commits. A merge produced obviously wrong output and you cannot explain where it went wrong.

Three different accidents, one root cause: you are memorizing commands without knowing which of Git's three data structures — objects, refs, and the index — each one touches.

Git's command surface is terrible to memorize. checkout used to do two unrelated jobs, "switch branches" and "restore files" (which is why v2.23 split it into switch and restore). reset changes how many of the three trees it moves based on a flag. The porcelain is a pile of historical accident.

The internals, by contrast, are startlingly simple. Git is a key-value store addressed by content hash, a set of 41-byte pointers into it, and a draft of the next commit. Once that clicks, commands stop being things you memorize and become things you derive.

This article takes Git's official documentation (as of Git 2.55) and the Documentation/ tree of the git/git repository as primary sources, and takes the internals apart. Every command output below was captured on git version 2.50.1. By the end you will be able to say, out loud, which tree an accident moved.


1. Git is a content-addressed key-value store

Experiment: you can compute the hash yourself

An object name (what people call the commit ID or SHA) is neither random nor sequential. It is computed deterministically from the content.

$ echo -n "what is up, doc?" | git hash-object --stdin
bd9dbf5aae1a3862dd1526723246b20206e5fc37

You can reproduce that value without Git. Git does not hash the content directly — it prepends a header of the form <type> <byte-length>\0 first.

$ printf 'blob 16\0what is up, doc?' | shasum
bd9dbf5aae1a3862dd1526723246b20206e5fc37  -

Identical. hash-object is nothing more than "prepend a header, take SHA-1."

Why the header? To put the type into the namespace. The same byte sequence stored as a blob and as a tree gets different object names. Type confusion is designed out.

Experiment: what it looks like on disk

With -w, the object is written into the object database.

$ printf 'hello\n' > a.txt
$ git hash-object -w a.txt
ce013625030ba8dba906f756967f9e9ca394464a

$ find .git/objects -type f
.git/objects/ce/013625030ba8dba906f756967f9e9ca394464a

The path splits the name: first two characters become the directory, the remaining 38 the filename. This keeps hundreds of thousands of files out of one directory, where many filesystems degrade sharply.

The content is zlib-compressed. Inflate it and the header is right there.

import zlib
data = open('.git/objects/ce/013625030ba8dba906f756967f9e9ca394464a', 'rb').read()
print(repr(zlib.decompress(data)))
# => b'blob 6\x00hello\n'

blob 6\0hello\n — exactly the bytes that were hashed, stored verbatim. Git's object store is, in this sense, remarkably literal.

Why this matters

This is where "Git stores snapshots, not diffs" comes from. Every commit points at a complete tree, but identical content produces an identical object name, so unchanged files never create new objects. A commit that touches one file out of ten thousand adds one blob and rebuilds only the trees along that path.

And "same content always has the same name" is an integrity check. Inflate a downloaded object, re-hash it, and both tampering and transfer corruption show up. That is what git fsck does.


2. The four object types

Git has exactly four.

TypeRoleContents
blobFile contentRaw bytes (no filename, no permissions)
treeDirectoryA list of "mode, type, object name, name"
commitSnapshot plus metadataRoot tree, parent commits, author/committer, message
tagAnnotated tagTarget object, tagger, message, optional signature

tree: this is where filenames live

Blobs have no name. Names and permissions belong to the tree.

$ git cat-file -p 2e81171448eb9f2ee3821e3d447aa6b2fe3ddba1
100644 blob ce013625030ba8dba906f756967f9e9ca394464a	a.txt

The mode is not a Unix permission bitmask; it is a small, fixed vocabulary.

ModeMeaning
100644Regular file
100755Executable file
120000Symbolic link
040000Subdirectory (tree)
160000gitlink (a submodule's commit reference)

gitformat-index(5) states that for the index's mode field, "only 0755 and 0644 are valid for regular files." Git records nothing about permissions beyond the executable bit. That is why chmod 600 is not version-controlled — and why you must never rely on Git to protect the permissions of a sensitive file.

commit: what makes the ID change

$ git cat-file -p 8994a20d11cbdb935ae7a857349e19a815e2e61e
tree 2e81171448eb9f2ee3821e3d447aa6b2fe3ddba1
author Demo <a@example.com> 1786806000 +0900
committer Demo <a@example.com> 1786806000 +0900

first

That is the entire commit object. A merge commit lists several parent lines; a signed commit carries a gpgsig line.

What matters is that the hash of this whole text is the commit ID. Three consequences follow directly.

  1. git commit --amend does not edit a commit. A different message means different bytes means a different ID. The original stays exactly where it was; only the ref moves.
  2. Rebase does not move history, it recreates it. Change a parent and the child's ID changes, and so does its child's, all the way down. That is why rebasing a shared branch is incompatible with everyone else's history.
  3. A signature is part of the commit. gpgsig lives inside the commit object, so the commit ID commits to the signature. You cannot swap a signature in afterwards.

Demo: building a commit out of plumbing alone

What the porcelain does internally is obvious once you reproduce it with plumbing.

$ git init -q -b main demo && cd demo
$ git config user.email a@example.com && git config user.name Demo

# 1. Put content into the object database (this creates a blob)
$ printf 'hello\n' > a.txt
$ git hash-object -w a.txt
ce013625030ba8dba906f756967f9e9ca394464a

# 2. Register it in the index (this is what git add does)
$ git update-index --add --cacheinfo 100644,ce013625030ba8dba906f756967f9e9ca394464a,a.txt
$ git ls-files --stage
100644 ce013625030ba8dba906f756967f9e9ca394464a 0	a.txt

# 3. Turn the index into a tree object
$ git write-tree
2e81171448eb9f2ee3821e3d447aa6b2fe3ddba1

# 4. Turn the tree into a commit object (no parent, so this is the root commit)
$ git commit-tree 2e81171448eb9f2ee3821e3d447aa6b2fe3ddba1 -m "first"
8994a20d11cbdb935ae7a857349e19a815e2e61e

# 5. Point the branch at that commit (only now is anything "committed")
$ git update-ref refs/heads/main 8994a20d11cbdb935ae7a857349e19a815e2e61e

git commit is steps 2 through 5 rolled into one command. Conversely, if that final update-ref fails, you are left with orphaned objects. That is why "I committed but it isn't in the log" is a refs problem far more often than an objects problem.


3. Refs: a branch is 41 bytes

A branch is a file

$ cat .git/refs/heads/main
8994a20d11cbdb935ae7a857349e19a815e2e61e
$ wc -c .git/refs/heads/main
      41 .git/refs/heads/main

Forty hex characters plus a newline: 41 bytes. That is a branch.

Which explains:

  • Creating a branch is instant. You write one 41-byte file.
  • Branches never make a repository heavier.
  • Deleting a branch does not delete history. You removed a pointer; the objects are still there (which is why you can get them back).

HEAD is slightly different — it is a symbolic ref.

$ cat .git/HEAD
ref: refs/heads/main

HEAD pointing at another ref is the normal state; HEAD holding an object name directly is a detached HEAD. The reason "commits made on a detached HEAD disappear" is that advancing HEAD updates no branch file, so the moment you switch branches nothing points at that commit any more (the reflog still does).

packed-refs and reftable

At tens of thousands of refs, "tens of thousands of 41-byte files" falls apart. The old answer is .git/packed-refs (all refs in one text file); Git's new answer is reftable.

git-init(1) is unambiguous:

--ref-format=<format> — the valid values are files, for loose files with packed-refs, which is the default, and reftable, for the reftable format.

And the official BreakingChanges.adoc announces that Git 3.0 will make reftable the default for new repositories, listing the reasons. In summary, five of them:

  1. On case-insensitive filesystems (Windows/macOS) you cannot hold refs/heads/Foo and refs/heads/foo at once. reftable does not encode ref names as filesystem paths, so the problem disappears.
  2. macOS normalizes Unicode in path names, so two differently-encoded ref names cannot coexist. Same resolution.
  3. Deleting a ref with the files backend requires rewriting the whole packed-refs file — dozens of megabytes, sometimes gigabytes, in large repositories. reftable uses tombstone markers, so no full rewrite.
  4. Writing multiple refs at once is not atomic with files. Other processes can observe an in-between state while a transaction is being committed.
  5. Writing many refs at once is slow, because each one becomes its own file. reftable outperforms files here "by multiple orders of magnitude," in the docs' own words.

reftable is a block-based binary format with prefix compression on ref names, maintained as an immutable stack of tables listed in .git/reftable/tables.list. Each update appends a new table, and swapping tables.list is the atomic commit. Readers search the stack newest-first.

Trying it is one command.

$ git init --ref-format=reftable myrepo
$ ls myrepo/.git/reftable/
0x000000000001-0x000000000001-0d8df5e0.ref
tables.list

$ cat myrepo/.git/reftable/tables.list
0x000000000001-0x000000000001-0d8df5e0.ref

The filename is ${min_update_index}-${max_update_index}-${random}.ref. Each update adds a table and appends a line to tables.list.

Migrating an existing repository is git refs migrate --ref-format=reftable (check first that no mirror or hook reads ref paths directly).

A remote-tracking branch is "what the remote looked like last time"

refs/remotes/origin/main is not the remote's current value. It is a snapshot from your last fetch. Holding that straight is the key to judging --force-with-lease correctly in the next section.

Applied: atomic ref updates in CI

When ref updates must be transactional, use git update-ref --stdin. Either all of them apply or none of them do.

# Update the release branch and the release tag indivisibly
$ git update-ref --stdin <<'EOF'
start
update refs/heads/release ceb1a2f9c8ec5f4e2a1d0b3c6e7f8a9b0c1d2e3f
create refs/tags/v1.4.0 ceb1a2f9c8ec5f4e2a1d0b3c6e7f8a9b0c1d2e3f
prepare
commit
EOF

The third argument of update <ref> <newvalue> [<oldvalue>] is the value you expect the ref to currently hold — optimistic locking, on the server side, detecting concurrent CI jobs stepping on each other. Requirements that branch protection rules cannot express ("move several refs together") can only be met safely this way.


4. The index: the third tree

What is inside .git/index

What people call "the staging area" is a single binary file, .git/index. Per gitformat-index(5):

  • A 12-byte header: the signature {'D','I','R','C'} (short for "dircache"), a version number (2, 3, and 4 are currently supported), and the number of entries
  • A run of sorted index entries
  • Extensions
  • A checksum over everything above

An entry holds more than an object name and a path. It holds the full result of stat(2) — ctime (seconds and nanoseconds), mtime (seconds and nanoseconds), dev, ino, mode, uid, gid, and file size.

That is where git status gets its speed. Git is not re-reading every file in the working tree; it compares the result of lstat() against the index record and skips reading content when they match, reading only what might have changed.

So when git status is slow, the cause is almost always in this layer. In order of effectiveness:

SymptomSetting that helpsWhat it does
Very many filesgit config core.untrackedCache truePer-directory cache for untracked-file discovery (the UNTR extension)
Virtual or network filesystemgit config core.fsmonitor trueUses OS change notifications to narrow the scan to what actually changed (the FSMN extension)
Monorepo, narrow work areacone-mode sparse-checkoutCollapses out-of-scope paths into directory-level entries (sparse index)

The sparse index is especially effective. gitformat-index(5) explains:

if sparse-checkout is enabled in cone mode (core.sparseCheckoutCone is enabled) and the extensions.sparseIndex extension is enabled, then the index may contain entries for directories outside of the sparse-checkout definition. These entries have mode 040000, include the SKIP_WORKTREE bit, and the path ends in a directory separator.

In other words, a 100,000-file subtree collapses into a single index entry. Since entry count is the cost of git status, this is not a linear improvement — it is an order-of-magnitude one.

The cache tree extension: why write-tree is fast

The index's TREE extension (the cache tree) caches the already-computed tree object name for each directory. Unchanged directories are immediately known not to need rebuilding, which is what keeps git write-tree — and therefore git commit — fast even in large repositories.

What a conflict really is: stages 1, 2, and 3

Normally index entries are at stage 0. When a merge conflicts, three generations occupy the same path.

$ git merge feature
Auto-merging f.txt
CONFLICT (content): Merge conflict in f.txt
Automatic merge failed; fix conflicts and then commit the result.

$ git ls-files -u
100644 83db48f84ec878fbfb30b46d16630e944e34f205 1	f.txt
100644 d791e9b8158e2be3a792fe1881d57828989a3449 2	f.txt
100644 00dbdcfd2b3a2c6a3a0facd9c753578741dc921e 3	f.txt
StageMeaningHow to read it
1Common ancestor (base)git show :1:f.txt
2Our side (ours / HEAD)git show :2:f.txt
3Their side (theirs / MERGE_HEAD)git show :3:f.txt

Knowing this turns conflict resolution from guessing into comparing.

# What did they actually change? Diff their side against the ancestor.
$ git diff $(git merge-base HEAD MERGE_HEAD) MERGE_HEAD -- f.txt

# Show all three generations (--conflict=diff3 includes the ancestor)
$ git checkout --conflict=diff3 -- f.txt

It also changes what git add means. git add discards stages 1/2/3 and writes a single stage 0 — it is a declaration that you resolved the conflict. That is exactly why typing git add . without reading the result is dangerous.


5. Deriving reset and restore from the three trees

Here are the three trees we have met.

TreeWhere it livesWhat it represents
HEAD.git/HEAD → branch → commit"The last state you committed"
Index.git/index"The state you intend to commit next"
Working treeThe filesystem"The state you are editing right now"

git reset is the command for choosing how many of these trees to roll back.

CommandHEADIndexWorking treeTypical use
git reset --soft <c>✅ movesRedo the last commit (changes stay staged)
git reset --mixed <c> (default)✅ moves✅ movesUnstage
git reset --hard <c>✅ moves✅ movesmovesRoll back completely (uncommitted changes are lost)
git restore --staged <path>✅ movesUnstage one file
git restore <path>✅ movesDiscard edits to one file
git switch <branch>✅ moves✅ moves✅ movesSwitch branches (aborts if changes would conflict)

Git v2.23 split switch and restore out of checkout precisely because those rows were all crammed into one command. In new documentation and scripts, use switch and restore.

"Only --hard destroys the working tree" — that single line prevents most accidents. Put the other way round: --soft and --mixed lose nothing. The objects remain and so does the reflog.


6. Inside a merge: merge-base and the ort strategy

A three-way merge is a comparison against the ancestor

Comparing two branches alone cannot tell you whether a line was added on one side or deleted on the other. So Git uses the common ancestor (merge base) as a third reference point.

$ git merge-base main feature
c3dcd69263c50d4a3fa0fac8428f07fdf1dd27ba

For each line:

  • Ancestor matches ours, theirs differs → take theirs
  • Ancestor matches theirs, ours differs → take ours
  • Both differ → conflict (three generations land in stages 1/2/3)

The default strategy, ort

Quoting merge-strategies(7) directly:

ort — This is the default merge strategy when pulling or merging one branch. This strategy can only resolve two heads using a 3-way merge algorithm. When there is more than one common ancestor that can be used for 3-way merge, it creates a merged tree of the common ancestors and uses that as the reference tree for the 3-way merge. […] The name for this algorithm is an acronym ("Ostensibly Recursive's Twin") and came from the fact that it was written as a replacement for the previous default algorithm, recursive.

"More than one common ancestor" is the criss-cross merge case. When branches merge into each other, the common ancestor is no longer unique, and ort builds a virtual base tree by recursively merging the ancestors. Most of the cases where your intuition about "why did this conflict / why didn't it" fails live right here.

On recursive, the same document is explicit:

This is now a synonym for ort. It was an alternative implementation until v2.49.0, but was redirected to mean ort in v2.50.0.

Older articles telling you to pass -s recursive are advising a no-op.

Behaviors and traps worth knowing

The documentation warns about one of its own behaviors:

With the strategies that use 3-way merge (including the default, ort), if a change is made on both branches, but later reverted on one of the branches, that change will be present in the merged result; some people find this behavior confusing. It occurs because only the heads and the merge base are considered when performing a merge, not the individual commits.

"A change I reverted came back through the merge" is specified behavior, not a bug. If you revert on a shared branch, the branch merging it may need its own revert.

Strategy options worth remembering:

OptionWhen to reach for it
-X ours / -X theirsAuto-resolve conflicting hunks in favor of one side (not the same as the -s ours strategy, which never looks at the other side at all)
-X renormalizeKills mass conflicts when the two branches disagree on line-ending normalization or clean filters
-X diff-algorithm=patienceAvoids mismerges caused by meaningless matching lines such as closing braces (ort defaults to histogram)
-X find-renames=<n>Tune the rename-detection threshold (rename detection is on by default)

And if you are resolving the same conflict over and over, enable rerere. It records your resolution and reapplies it when the same conflict recurs — a large win when you repeatedly rebase a long-lived branch.

$ git config --global rerere.enabled true

7. What rebase and force push really do, and how to use --force-with-lease

Rebase creates new commits

As section 2 showed, a commit ID is the hash of content that includes the parent. So rebase does not move commits; it creates new ones carrying the same changes. The originals become unreachable and live on in the reflog.

That is the real reason behind "don't rebase a shared branch." Other people hold the old IDs you just abandoned, and your push is incompatible with them.

How --force-with-lease works, and the limit the docs admit

git-push(1) rewards careful reading.

This option overrides this restriction if the current value of the remote ref is the expected value. git push fails otherwise. […] It is like taking a "lease" on the ref without explicitly locking it, and the remote ref is updated only if the "lease" is still valid.

The question is where the expected value comes from.

--force-with-lease alone, without specifying the details, will protect all remote refs that are going to be updated by requiring their current value to be the same as the remote-tracking branch we have for them.

So the expected value is refs/remotes/origin/*. And the documentation states the following as a general note on safety:

supplying this option without an expected value, i.e. as --force-with-lease or --force-with-lease=<refname> interacts very badly with anything that implicitly runs git fetch on the remote to be pushed to in the background, e.g. git fetch origin on your repository in a cronjob. […] The protection it offers over --force is ensuring that subsequent changes your work wasn't based on aren't clobbered, but this is trivially defeated if some background process is updating refs in the background.

Editor extensions and IDEs that fetch periodically are common. So automation (CI, scripts) should use the explicit form.

# Good: you declare the remote value you expect
$ EXPECTED=$(git rev-parse refs/remotes/origin/main)
$ git push --force-with-lease="main:${EXPECTED}" origin main

# Stronger: also require that the remote tip is in your own reflog
$ git push --force-with-lease --force-if-includes origin main

--force-if-includes is a complementary check. Per the docs:

This option enables a check that verifies if the tip of the remote-tracking ref is reachable from one of the "reflog" entries of the local branch based in it for a rewrite. The check ensures that any updates from the remote have been incorporated locally by rejecting the forced update if that is not the case.

There is a caveat, also stated: passing it without --force-with-lease, or alongside --force-with-lease=<refname>:<expect>, is a no-op. The first has nothing to build on; the second already declares the expectation explicitly, so no further inference is needed.

Where each option belongs in practice

SituationRecommendation
Your own topic branchgit push --force-with-lease
Forced updates from CI/automation--force-with-lease=<ref>:<expect> with an explicit expectation
Shared or protected branchesForbid force updates on the server (branch protection rules)
A shared branch whose history must be fixedOpen a new branch and a PR; do not force push

Rather than banning --force outright, block it server-side and make --force-with-lease the client default. This is a place to rely on mechanism, not human discipline.


8. Packfiles: how transfer and storage actually work

From loose objects to packs

Everything so far — one object, one file — is the loose object format. It becomes inefficient in bulk, so Git periodically consolidates into packfiles.

# 13 loose objects
$ git count-objects -vH
count: 13
size: 52.00 KiB
in-pack: 0
packs: 0
size-pack: 0 bytes

$ git gc -q

# One pack, and much smaller
$ git count-objects -vH
count: 0
size: 0 bytes
in-pack: 13
packs: 1
size-pack: 2.16 KiB

52.00 KiB → 2.16 KiB. This is a three-commit toy repository, so do not generalize that ratio — but the underlying property holds everywhere: loose objects are rounded up to the filesystem's block size and therefore occupy far more space than their real size.

The format (gitformat-pack(5))

A .pack file is structured as:

  • A 4-byte signature {'P','A','C','K'}
  • A 4-byte version number (Git accepts version 2 or 3 but generates version 2 only)
  • A 4-byte object count
  • A run of object entries
  • A trailing checksum over the whole pack

There are seven entry types (one of which is reserved).

ValueType
1OBJ_COMMIT
2OBJ_TREE
3OBJ_BLOB
4OBJ_TAG
6OBJ_OFS_DELTA (base identified by relative offset within the same pack)
7OBJ_REF_DELTA (base identified by object name)

Types 6 and 7 are delta compression — the physical answer to "how can snapshots not explode in size." Similar objects are stored as a base plus delta instructions, and zlib is applied on top.

The actual delta chains are visible with verify-pack.

$ git verify-pack -v .git/objects/pack/pack-*.idx
719d02e...  commit 265 184 12
766195b...  commit  67  78 196 1 719d02e...
faf9142...  commit  19  30 274 2 766195b...

The rightmost number is the delta depth, followed by the base object. 766195b is a depth-1 delta against 719d02e; faf9142 is depth 2 on top of that. A 67-byte commit is stored in 30 bytes.

The .idx file and the multi-pack index

Scanning a pack linearly would defeat the purpose, so each pack ships an .idx. The v2 layout is:

  • The magic number \377tOc and version 2
  • A 256-entry fan-out table (cumulative counts by the first byte of the object name)
  • A sorted object-name table
  • A CRC32 per object (new in v2 — corruption detection for transfer and storage)
  • A 4-byte offset table (offsets past 2 GiB spill into an 8-byte large-offset table)
  • The pack checksum and the index's own checksum

The fan-out table narrows the search space to 1/256 using a single byte; a binary search follows. Even at millions of objects, that is a few dozen comparisons.

When there are many packs, searching across .idx files becomes the bottleneck, which is what the multi-pack-index (MIDX) solves. Its signature is {'M','I','D','X'}, and it is organized into chunks such as PNAM (pack names), OIDF (fan-out), OIDL (name lookup), and OOFF (offsets).

commit-graph: an index for history traversal

Reading parent-child relationships out of commit objects on every traversal makes git log --graph and reachability checks slow. The commit-graph file indexes them separately.

Add --changed-paths and Git also writes Bloom filters of changed paths.

With the --changed-paths option, compute and write information about the paths changed between a commit and its first parent. This operation can take a while on large repositories. It provides significant performance gains for getting history of a directory or a file with git log -- <path>.

In a monorepo where git log -- path/to/file crawls, this is usually the single highest-impact change available.


9. Practical answers for huge repositories

Partial clone: fetch objects lazily

The official design note states the motivation plainly:

During clone and fetch operations, Git downloads the complete contents and history of the repository. […] For extremely large repositories, clones can take hours (or days) and consume 100+GiB of disk space.

A partial clone defers fetching objects until they are needed. A remote that can supply them later is called a promisor remote.

# Clone without historical blobs (fetched only when a file is actually opened)
$ git clone --filter=blob:none https://github.com/org/repo.git

# Without trees either (for CI builds and similar, where history traversal is not needed)
$ git clone --filter=tree:0 --depth=1 https://github.com/org/repo.git

There is an important precondition, also stated officially:

Use of partial clone requires that the user be online and the origin remote or other promisor remotes be available for on-demand fetching of missing objects.

Offline work or a flaky network turns an innocuous git log -p into a storm of extra fetches, making things slower. This is a choice per use case, not something to impose on everyone.

sparse-checkout (cone mode)

sparse-checkout narrows the working-tree side. Cone mode trades path-level flexibility for directory-level restriction and speed, and combines with the sparse index described earlier.

$ git sparse-checkout set --cone apps/web packages/ui
$ git config core.sparseCheckoutCone true

git maintenance: make upkeep a mechanism

Nobody keeps running git gc by hand. git maintenance is a per-task scheduler.

$ git maintenance start   # register with the OS scheduler and run periodically

The default schedule for the incremental strategy, per git-maintenance(1):

TaskWhat it doesDefault schedule
gcRepack everything into a single packDisabled (in the incremental strategy)
commit-graphIncrementally update and verify the commit-graphHourly
prefetchPre-fetch the latest objects from all remotes into refs/prefetch/Hourly
loose-objectsMove loose objects into packs in batchesDaily
incremental-repackRepack using the multi-pack-indexDaily
pack-refsCollect loose refs into a single fileNot scheduled (must be enabled explicitly)
reflog-expireDelete expired reflog entriesNot scheduled
rerere-gcClean up the rerere cacheNot scheduled
worktree-pruneDelete stale or broken worktreesNot scheduled

The incremental strategy actually schedules only those first five (gc is explicitly listed as "disabled"). pack-refs and below exist as tasks but are not part of the default schedule. Enable them explicitly if you need them.

# Put pack-refs on a weekly schedule in a ref-heavy repository
$ git config maintenance.pack-refs.enabled true
$ git config maintenance.pack-refs.schedule weekly

gc being disabled in the incremental strategy is deliberate. A full repack is expensive and it deletes unreachable objects, so day-to-day upkeep is better served by the incremental loose-objects and incremental-repack tasks.

Practical CI patterns

Git in CI has different requirements from Git on a developer's machine: usually you need the working tree and not the history.

# GitHub Actions: jobs that don't need history at all (build, test)
- uses: actions/checkout@v5
  with:
    fetch-depth: 1

# Jobs that only need to know which files changed (diff-based lint, impact analysis)
# → history metadata is required, past file contents are not
- uses: actions/checkout@v5
  with:
    fetch-depth: 0
    filter: blob:none

fetch-depth: 0 (full history) is the most expensive setting by default. Adding filter: blob:none keeps all commits and trees while skipping historical file contents, cutting transfer by an order of magnitude on large repositories.


10. Integrity and the future: SHA-1, SHA-256, and Git 3.0

Today's SHA-1 is not plain SHA-1

hash-function-transition records:

Git v2.13.0 and later subsequently moved to a hardened SHA-1 implementation by default, which isn't vulnerable to the SHAttered attack.

That is sha1dc (SHA-1 with collision detection): it recognizes the bit patterns characteristic of collision attacks and rejects them. So "the SHAttered PDFs can be smuggled into Git" does not apply to current Git.

The same document immediately adds that SHA-1 is still weak.

SHA-256 works, but is not yet a production choice

$ git init --object-format=sha256 myrepo

git-init(1) is candid:

The valid values are sha1 and (if enabled) sha256. sha1 is the default. Note: At present, there is no interoperability between SHA-256 repositories and SHA-1 repositories.

And hash-function-transition adds an operational warning:

Until Git protocol gains SHA-256 support, using SHA-256 based storage on public-facing Git servers is strongly discouraged.

Conclusion: starting a new project on SHA-256 today is premature. Waiting for hosting providers and the toolchain (JGit, libgit2, gitoxide) to catch up is the correct call.

What changes in Git 3.0

The official BreakingChanges.adoc lists five headline changes (with no release date announced).

ChangeDetailPractical impact
Default hashsha1sha256New repositories only; the docs state there is no plan to deprecate sha1
Default ref storagefilesreftableScripts and hooks that read .git/refs/* directly will break
Default branch namemastermainAssumptions baked into CI config and docs
Build requirementRust becomes mandatory (2.55 default-enables it in both build systems; 3.0 requires it)Affects self-built and embedded distributions
safe.bareRepositoryallexplicitA security hardening: implicit discovery of bare repositories is refused

The last one deserves the official explanation in full:

It is all too easy for an attacker to trick a user into cloning a repository that contains an embedded bare repository with malicious hooks configured. If the user enters that subdirectory and runs any Git command, Git discovers the bare repository and the hooks fire. The user does not even need to run a Git command explicitly: many shell prompts run git status in the background to display branch and dirty state information, and git status in turn may invoke the fsmonitor hook if so configured, making the user vulnerable the moment they cd into the directory.

Arbitrary code execution from a cd is something every day-to-day Git user should know about now. You do not have to wait for Git 3.0.

$ git config --global safe.bareRepository explicit

Bare repositories specified explicitly via --git-dir or GIT_DIR keep working, so the operational side effects are close to zero. If you ever clone repositories you don't fully trust (which is nearly everyone), this is safe to turn on today.


11. Production checklist

Turning the internals into settings and practice.

Safe for everyone

# Prevent implicit hook execution via a malicious bare repository (Git 3.0's default, early)
git config --global safe.bareRepository explicit

# Record and replay conflict resolutions
git config --global rerere.enabled true

# A saner push default (understanding the limits from §7)
git config --global push.default simple

# Detect corrupt objects on the way in and out
git config --global transfer.fsckObjects true
git config --global fetch.fsckObjects true
git config --global receive.fsckObjects true

Effective on large repositories

# Cache untracked-file discovery (the UNTR extension)
git config core.untrackedCache true

# Cut stat traversal using filesystem change notifications (the FSMN extension)
git config core.fsmonitor true

# Register periodic maintenance with the OS scheduler
git maintenance start

What to look for in review

ConcernHow to checkWhy
Large binaries in historyInspect the largest entries via git count-objects -vH / git verify-pack -vOnce in, they ride along in every clone forever (deltas don't help)
Secrets committedScan the entire history, not the tipThe objects remain, so deleting them in a later commit achieves nothing
Can anyone force push a shared branch?Server-side branch protectionClient settings depend on discipline
Is CI checking out more than it needs?The fetch-depth and filter settingsA full-history clone costs jobs × repository size in transfer
Are hooks trustworthy?core.hooksPath and safe.bareRepositoryHooks are arbitrary code execution

12. Conclusion: from memorizing to deriving

Boiled down, Git's internals are three lines.

  1. The object database is an append-only key-value store keyed by content hash. Nothing you write disappears (until GC). That is why recovery is possible.
  2. A ref is a 41-byte pointer into that store. Cheap branching, rebase-as-recreation, and the danger of force push all follow from this one fact.
  3. The index is a draft of the next commit. It doubles as a stat cache, which is why status is fast, and it holds three generations at once (stages 1/2/3) during a conflict.

With those three in hand, you know which tree you are about to destroy before you press enter. Git accidents are, almost without exception, accidents in which only the working tree is lost — the objects are still there. And the set of commands that destroy the working tree is small.

The natural next step is the recovery procedures for when something is already lost: how long the reflog lives, the grace period for unreachable objects (two weeks by default), cruft packs, and git fsck --lost-found. Knowing what survives and for how long is what keeps you calm when it happens.

Frequently asked questions

Does Git store diffs or snapshots?
The storage model is snapshots. Every commit points at a full tree, and file content is stored whole as blob objects. But identical content gets an identical object name, so unchanged files never create a new object — they share the existing blob. On top of that, packing applies delta compression (OBJ_OFS_DELTA / OBJ_REF_DELTA), so on disk the data is held in something close to diff form. The accurate summary is: snapshots as the logical model, deltas as the physical representation.
What is the difference between git reset --soft, --mixed, and --hard, in one line?
The number of trees each one moves. --soft moves only HEAD, leaving the index and the working tree alone (use it to redo a commit). --mixed (the default) moves HEAD and the index but leaves the working tree (use it to unstage). --hard moves all three, so any uncommitted change is lost. If you have changes you care about, run git stash before --hard.
Is git push --force-with-lease safer than --force?
Conditionally. --force-with-lease only lets the update through when the remote's current value matches your remote-tracking branch. But the official docs state plainly that if an editor or a cron job runs git fetch in the background, this protection is 'trivially defeated'. From CI or scripts, use the explicit form --force-with-lease=<refname>:<expect>, or add --force-if-includes.
SHA-1 has been broken. Is Git unsafe?
Since v2.13.0 Git has used a hardened SHA-1 (sha1dc) by default, which detects known collision attacks such as SHAttered. That said, the official BreakingChanges file calls SHA-1 'still weak' and announces that Git 3.0 will change the default hash for new repositories to SHA-256. For now, SHA-256 repositories cannot interoperate with SHA-1 repositories, and the docs strongly discourage SHA-256 on public-facing servers until the wire protocol supports it — so this is not the moment to migrate production.
Clone and status are slow on a huge repository. What should I do first?
Measure, then apply three things in order of impact. (1) If the object volume of history is the problem, use a partial clone (--filter=blob:none) so blobs are fetched on demand. (2) If the number of files in the working tree is the problem, use cone-mode sparse-checkout with the sparse index to shrink the index itself. (3) If traversal cost is the problem, run git maintenance start so commit-graph, prefetch, loose-objects, and incremental-repack run on a schedule. In CI, combining fetch-depth with filter gives the best return and can cut per-job transfer by an order of magnitude.

References

友田

友田 陽大

Developer of a METI Minister's Award–winning product. With TypeScript + Python + AWS, I deliver SaaS, industry DX, and production-grade generative AI (RAG) end to end — from requirements to infrastructure and operations — single-handedly.

I can take on the implementation from this article as an engagement

Repository operations & CI pipeline design and speed-up

Clone, status and CI got slow as the repository grew; the monorepo split keeps stalling; history-rewrite accidents keep repeating — the cause is almost always Git's internals (object volume, ref count, index size). I measure first, then decide whether partial clone, sparse-checkout, commit-graph and git maintenance actually apply, and design and implement the result down to branch strategy and protection rules.

Available for both project-based (contract) and advisory engagements. Start with a free 30-minute consult.

最短ルート:カレンダーから直接予約

相談内容が固まっている方は、フォーム送信よりその場で日程を確定する方がスムーズです。下記から空き時間をお選びください。

  • 30分のオンライン無料相談
  • Google Meet / Zoom / Microsoft Teams
  • NDA 商談前締結可・無理な営業はいたしません
無料相談の空き枠を予約する

Also worth reading