mirror-linux/mm
Balbir Singh 368076f52e mm/huge_memory: add device-private THP support to PMD operations
Extend core huge page management functions to handle device-private THP
entries.  This enables proper handling of large device-private folios in
fundamental MM operations.

The following functions have been updated:

- copy_huge_pmd(): Handle device-private entries during fork/clone
- zap_huge_pmd(): Properly free device-private THP during munmap
- change_huge_pmd(): Support protection changes on device-private THP
- __pte_offset_map(): Add device-private entry awareness

Link: https://lkml.kernel.org/r/20251001065707.920170-4-balbirs@nvidia.com
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Balbir Singh <balbirs@nvidia.com>
Acked-by: Zi Yan <ziy@nvidia.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: Ying Huang <ying.huang@linux.alibaba.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Lyude Paul <lyude@redhat.com>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: David Airlie <airlied@gmail.com>
Cc: Simona Vetter <simona@ffwll.ch>
Cc: Ralph Campbell <rcampbell@nvidia.com>
Cc: Mika Penttilä <mpenttil@redhat.com>
Cc: Francois Dugast <francois.dugast@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2025-11-24 15:08:48 -08:00
..
damon mm/damon/tests/core-kunit: remove DAMON_MIN_REGION redefinition 2025-11-20 13:44:01 -08:00
kasan kasan: cleanup of kasan_enabled() checks 2025-11-16 17:28:01 -08:00
kfence kfence: drop nth_page() usage 2025-09-21 14:22:09 -07:00
kmsan kmsan: remove hard-coded GFP_KERNEL flags 2025-11-16 17:27:54 -08:00
Kconfig mm: handle poisoning of pfn without struct pages 2025-11-16 17:28:29 -08:00
Kconfig.debug mm: rename GENERIC_PTDUMP and PTDUMP_CORE 2025-03-17 00:05:32 -07:00
Makefile mm/hugetlb: extract sysctl into hugetlb_sysctl.c 2025-11-20 13:43:57 -08:00
backing-dev.c fuse update for 6.18 2025-10-03 12:48:18 -07:00
balloon_compaction.c mm/migrate: fix NULL movable_ops if CONFIG_ZSMALLOC=m 2025-08-19 16:35:57 -07:00
bootmem_info.c mm/sparse: allow for alternate vmemmap section init at boot 2025-03-16 22:06:27 -07:00
cma.c mm/cma: refuse handing out non-contiguous page ranges 2025-09-21 14:22:06 -07:00
cma.h mm: cma: set early_pfn and bitmap as a union in cma_memrange 2025-05-22 14:55:36 -07:00
cma_debug.c mm: cma: simplify cma_maxchunk_get() 2025-07-24 19:12:36 -07:00
cma_sysfs.c mm/cma: export total and free number of pages for CMA areas 2025-03-16 22:06:24 -07:00
compaction.c mm/compaction: fix low_pfn advance on isolating hugetlb 2025-09-28 11:51:29 -07:00
debug.c mm: constify __dump_folio() arguments 2025-11-20 13:43:57 -08:00
debug_page_alloc.c mm/debug_page_alloc: improve error message for invalid guardpage minorder 2025-05-12 23:50:38 -07:00
debug_page_ref.c
debug_vm_pgtable.c treewide: include linux/pgalloc.h instead of asm/pgalloc.h 2025-11-16 17:28:25 -08:00
dmapool.c docs: dma-api: replace consistent with coherent 2025-07-01 13:25:36 -06:00
dmapool_test.c
early_ioremap.c
execmem.c mm: remove PMD alignment constraint in execmem_vmalloc() 2025-09-28 11:51:31 -07:00
fadvise.c
fail_page_alloc.c
failslab.c
filemap.c Merge branch 'mm-hotfixes-stable' into mm-stable in order to merge 2025-11-24 15:07:34 -08:00
folio-compat.c mm: Remove grab_cache_page_write_begin() 2025-03-04 17:02:25 +00:00
gup.c mm: replace READ_ONCE() with standard page table accessors 2025-11-16 17:27:56 -08:00
gup_test.c
gup_test.h
highmem.c mm: constify highmem related functions for improved const-correctness 2025-09-21 14:22:15 -07:00
hmm.c mm: replace READ_ONCE() with standard page table accessors 2025-11-16 17:27:56 -08:00
huge_memory.c mm/huge_memory: add device-private THP support to PMD operations 2025-11-24 15:08:48 -08:00
hugetlb.c mm/hugetlb: extract sysctl into hugetlb_sysctl.c 2025-11-20 13:43:57 -08:00
hugetlb_cgroup.c page_counter: track failcnt only for legacy cgroups 2025-03-17 00:05:35 -07:00
hugetlb_cma.c mm: hugetlb: directly pass order when allocate a hugetlb folio 2025-09-21 14:22:11 -07:00
hugetlb_cma.h mm: hugetlb: directly pass order when allocate a hugetlb folio 2025-09-21 14:22:11 -07:00
hugetlb_internal.h mm/hugetlb: extract sysctl into hugetlb_sysctl.c 2025-11-20 13:43:57 -08:00
hugetlb_sysctl.c mm/hugetlb: extract sysctl into hugetlb_sysctl.c 2025-11-20 13:43:57 -08:00
hugetlb_sysfs.c mm/hugetlb: extract sysfs into hugetlb_sysfs.c 2025-11-20 13:43:57 -08:00
hugetlb_vmemmap.c treewide: include linux/pgalloc.h instead of asm/pgalloc.h 2025-11-16 17:28:25 -08:00
hugetlb_vmemmap.h mm/hugetlb: do pre-HVO for bootmem allocated pages 2025-03-16 22:06:29 -07:00
hwpoison-inject.c mm/hwpoison: decouple hwpoison_filter from mm/memory-failure.c 2025-09-21 14:22:21 -07:00
init-mm.c mm: replace vm_lock and detached flag with a reference count 2025-03-16 22:06:20 -07:00
internal.h mm/madvise: allow guard page install/remove under VMA lock 2025-11-20 13:43:59 -08:00
interval_tree.c
ioremap.c mm/ioremap: pass pgprot_t to ioremap_prot() instead of unsigned long 2025-03-16 22:06:23 -07:00
khugepaged.c mm: set the VM_MAYBE_GUARD flag on guard region install 2025-11-20 13:43:58 -08:00
kmemleak.c mm: fix possible deadlock in kmemleak 2025-09-01 17:11:37 -07:00
ksm.c ksm: replace function unmerge_ksm_pages with break_ksm 2025-11-16 17:28:28 -08:00
list_lru.c mm, list_lru: refactor the locking code 2025-07-09 22:41:56 -07:00
maccess.c mm: unexport globally copy_to_kernel_nofault 2025-07-09 22:42:22 -07:00
madvise.c mm/madvise: allow guard page install/remove under VMA lock 2025-11-20 13:43:59 -08:00
mapping_dirty_helpers.c mm/dirty: replace READ_ONCE() with pudp_get() 2025-11-16 17:27:58 -08:00
memblock.c kho: replace kho_preserve_phys() with kho_preserve_pages() 2025-10-07 13:48:55 -07:00
memcontrol-v1.c mm/memcg: v1: account event registrations and drop world-writable cgroup.event_control 2025-09-21 14:22:26 -07:00
memcontrol-v1.h memcg: move do_memsw_account() to CONFIG_MEMCG_V1 2025-03-21 22:03:11 -07:00
memcontrol.c mm: memcg: dump memcg protection info on oom or alloc failures 2025-11-20 13:43:59 -08:00
memfd.c mm/memfd: fix information leak in hugetlb folios 2025-11-24 14:25:17 -08:00
memory-failure.c mm: handle poisoning of pfn without struct pages 2025-11-16 17:28:29 -08:00
memory-tiers.c mm: fix some typos in mm module 2025-11-16 17:27:52 -08:00
memory.c mm: introduce copy-on-fork VMAs and make VM_MAYBE_GUARD one 2025-11-20 13:43:58 -08:00
memory_hotplug.c memory_hotplug: optimise try_offline_memory_block() 2025-11-20 13:43:57 -08:00
mempolicy.c mm: mprotect: convert to folio_can_map_prot_numa() 2025-11-16 17:28:03 -08:00
mempool.c mm: mempool: fix crash in mempool_free() for zero-minimum pools 2025-08-02 12:06:13 -07:00
memremap.c mm/zone_device: rename page_free callback to folio_free 2025-11-24 15:08:47 -08:00
memtest.c
migrate.c migrate: optimise alloc_migration_target() 2025-11-20 13:43:57 -08:00
migrate_device.c treewide: remove MIGRATEPAGE_SUCCESS 2025-09-13 16:54:50 -07:00
mincore.c mm, swap: use unified helper for swap cache look up 2025-09-21 14:22:22 -07:00
mlock.c mm: update vma_modify_flags() to handle residual flags, document 2025-11-20 13:43:58 -08:00
mm_init.c drivers/base/node: fold register_node() into register_one_node() 2025-11-16 17:28:02 -08:00
mm_slot.h
mmap.c mm: use vma_start_write_killable() in dup_mmap() 2025-11-20 13:43:59 -08:00
mmap_lock.c Merge branch 'mm-hotfixes-stable' into mm-stable in order to merge 2025-11-24 15:07:34 -08:00
mmu_gather.c treewide: include linux/pgalloc.h instead of asm/pgalloc.h 2025-11-16 17:28:25 -08:00
mmu_notifier.c Update Christoph's Email address and make it consistent 2025-05-12 23:50:31 -07:00
mmzone.c mm: introduce memdesc_flags_t 2025-09-13 16:55:07 -07:00
mprotect.c mm: update vma_modify_flags() to handle residual flags, document 2025-11-20 13:43:58 -08:00
mremap.c treewide: include linux/pgalloc.h instead of asm/pgalloc.h 2025-11-16 17:28:25 -08:00
mseal.c mm: update vma_modify_flags() to handle residual flags, document 2025-11-20 13:43:58 -08:00
msync.c
nommu.c mm/nommu: convert kobjsize() to folios 2025-09-13 16:54:46 -07:00
numa.c mm/numa: remove unnecessary local variable in alloc_node_data() 2025-05-12 23:50:38 -07:00
numa_emulation.c mm: numa,memblock: Use SZ_1M macro to denote bytes to MB conversion 2025-08-20 16:31:23 +03:00
numa_memblks.c mm: numa,memblock: Use SZ_1M macro to denote bytes to MB conversion 2025-08-20 16:31:23 +03:00
oom_kill.c mm: memcg: dump memcg protection info on oom or alloc failures 2025-11-20 13:43:59 -08:00
page-writeback.c fuse update for 6.18 2025-10-03 12:48:18 -07:00
page_alloc.c mm: memcg: dump memcg protection info on oom or alloc failures 2025-11-20 13:43:59 -08:00
page_counter.c page_counter: track failcnt only for legacy cgroups 2025-03-17 00:05:35 -07:00
page_ext.c mm,page_ext: derive the node from the pfn 2025-07-13 16:38:16 -07:00
page_frag_cache.c
page_idle.c mm: always call rmap_walk() on locked folios 2025-11-16 17:28:00 -08:00
page_io.c mm, swap: tidy up swap device and cluster info helpers 2025-09-21 14:22:23 -07:00
page_isolation.c mm/page_isolation: drop __folio_test_movable() check for large folios 2025-07-13 16:38:29 -07:00
page_owner.c mm/page_owner: simplify zone iteration logic in init_early_allocated_pages() 2025-11-16 17:28:01 -08:00
page_poison.c
page_reporting.c
page_reporting.h
page_table_check.c mm/page_table_check: Batch-check pmds/puds just like ptes 2025-05-09 13:43:07 +01:00
page_vma_mapped.c mm/page_vma_mapped: track if the page is mapped across page table boundary 2025-09-28 11:51:29 -07:00
pagewalk.c mm/madvise: allow guard page install/remove under VMA lock 2025-11-20 13:43:59 -08:00
percpu-internal.h
percpu-km.c mm/mm/percpu-km: drop nth_page() usage within single allocation 2025-09-21 14:22:04 -07:00
percpu-stats.c mm: remove outdated filename comment in percpu-stats.c 2025-07-13 16:38:23 -07:00
percpu-vm.c kmsan: remove hard-coded GFP_KERNEL flags 2025-11-16 17:27:54 -08:00
percpu.c percpu: fix race on alloc failed warning limit 2025-09-08 23:45:10 -07:00
pgalloc-track.h
pgtable-generic.c mm/huge_memory: add device-private THP support to PMD operations 2025-11-24 15:08:48 -08:00
process_vm_access.c
pt_reclaim.c treewide: include linux/pgalloc.h instead of asm/pgalloc.h 2025-11-16 17:28:25 -08:00
ptdump.c mm/ptdump: replace READ_ONCE() with standard page table accessors 2025-11-16 17:27:52 -08:00
readahead.c readahead: add trace points 2025-09-21 14:22:28 -07:00
rmap.c mm/zone_device: support large zone device private folios 2025-11-24 15:08:47 -08:00
rodata_test.c
secretmem.c mm: add vma_desc_size(), vma_desc_pages() helpers 2025-11-16 17:28:11 -08:00
shmem.c mm: shmem/tmpfs hugepage defaults config choice 2025-11-16 17:28:23 -08:00
shmem_quota.c
show_mem.c mm: re-enable kswapd when memory pressure subsides or demotion is toggled 2025-09-21 14:22:29 -07:00
shrinker.c
shrinker_debug.c mm/shrinker: fix name consistency issue in shrinker_debugfs_rename() 2025-03-17 00:05:40 -07:00
shuffle.c
shuffle.h
slab.h Summary of significant series in this pull request: 2025-10-02 18:18:33 -07:00
slab_common.c mm: fix some typos in mm module 2025-11-16 17:27:52 -08:00
slub.c mm: remove reference to destructor in comment in calculate_sizes() 2025-11-16 17:28:15 -08:00
sparse-vmemmap.c mm: replace READ_ONCE() with standard page table accessors 2025-11-16 17:27:56 -08:00
sparse.c mm: introduce memdesc_nid() 2025-09-13 16:55:07 -07:00
swap.c mm: lru_add_drain_all() do local lru_add_drain() first 2025-09-21 14:22:32 -07:00
swap.h mm/migrate, swap: drop usage of folio_index 2025-11-16 17:28:20 -08:00
swap_cgroup.c mm: swap_cgroup: remove double initialization of locals 2025-03-17 22:06:58 -07:00
swap_state.c mm, swap: fix potential UAF issue for VMA readahead 2025-11-15 10:52:02 -08:00
swap_table.h mm, swap: use a single page for swap table when the size fits 2025-09-21 14:22:25 -07:00
swapfile.c Merge branch 'mm-hotfixes-stable' into mm-stable in order to merge 2025-11-24 15:07:34 -08:00
truncate.c mm/truncate: unmap large folio on split failure 2025-11-09 21:19:43 -08:00
usercopy.c mm: security: Check early if HARDENED_USERCOPY is enabled 2025-02-28 11:51:31 -08:00
userfaultfd.c mm/userfaultfd: don't lock anon_vma when performing UFFDIO_MOVE 2025-11-16 17:28:00 -08:00
util.c mm: add ability to take further action in vm_area_desc 2025-11-16 17:28:12 -08:00
vma.c mm: implement sticky VMA flags 2025-11-20 13:43:58 -08:00
vma.h mm: implement sticky VMA flags 2025-11-20 13:43:58 -08:00
vma_exec.c mm/vma: use vmg->target to specify target VMA for new VMA merge 2025-07-09 22:42:11 -07:00
vma_init.c Summary of significant series in this pull request: 2025-10-02 18:18:33 -07:00
vma_internal.h
vmalloc.c mm/vmalloc: request large order pages from buddy allocator 2025-11-16 17:28:15 -08:00
vmpressure.c memcg: convert memcg->socket_pressure to u64 2025-07-24 19:12:32 -07:00
vmscan.c mm: vmscan: simplify the folio refcount check in pageout() 2025-11-20 13:43:56 -08:00
vmstat.c mm: vmstat: correct the comment above preempt_disable_nested() 2025-11-20 13:43:59 -08:00
workingset.c mm: introduce memdesc_flags_t 2025-09-13 16:55:07 -07:00
zpdesc.h mm: zpdesc: minor naming and comment corrections 2025-09-21 14:21:59 -07:00
zsmalloc.c mm: remove unused zpool layer 2025-09-21 14:21:59 -07:00
zswap.c mm/zswap: s/red-black tree/xarray/ 2025-11-16 17:27:57 -08:00