Skip to content

llama : expose C API to get layer device type #14358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

llama : expose C API to get layer device type #14358

wants to merge 1 commit into from
+6 −0

Conversation

okaris
Copy link

@okaris okaris commented Jun 24, 2025

llama : expose C API to get layer device type

Adds llama_model_dev_layer(model, il) to retrieve the backend device type (CPU, GPU, ACCEL) for a given layer. This allows consumers (e.g. Python bindings) to inspect layer placement without exposing internal structures.

Implements an int32_t-based interface matching the pattern of existing public API functions such as llama_model_n_layer.

Notes:

  • Follows naming convention: llama_model_dev_layer
  • Pure C-compatible API (int32_t return)
  • No changes to third-party deps, no extra headers
  • Scoped, focused, standalone change
  • Adheres to existing formatting and structure
  • Useful for inspection/debugging/perf profiling via bindings
@okaris okaris force-pushed the master branch 2 times, most recently from ac6909a to 1db1d5b Compare June 24, 2025 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
1 participant
@okaris