Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -4,8 +4,155 @@ tags:
|
|
4 |
- lighter
|
5 |
- model_hub_mixin
|
6 |
- pytorch_model_hub_mixin
|
|
|
|
|
|
|
7 |
---
|
8 |
|
9 |
-
|
10 |
-
|
11 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4 |
- lighter
|
5 |
- model_hub_mixin
|
6 |
- pytorch_model_hub_mixin
|
7 |
+
language: en
|
8 |
+
license: apache-2.0
|
9 |
+
arxiv: 2501.09001
|
10 |
---
|
11 |
|
12 |
+
# CT-FM Feature Extractor
|
13 |
+
|
14 |
+
This model is a feature extractor for CT-FM, a model for whole-body segmentation. The feature extractor is based on SegResNet, a 3D U-Net variant.
|
15 |
+
|
16 |
+
If you want to just load the model and fine-tune, ignore the feature extraction workflow.
|
17 |
+
|
18 |
+
## Running instructions
|
19 |
+
|
20 |
+
|
21 |
+
# Whole Body Segmentation Inference
|
22 |
+
|
23 |
+
This notebook demonstrates how to:
|
24 |
+
1. Load a pre-trained whole body segmentation model from HuggingFace Hub
|
25 |
+
2. Set up preprocessing and postprocessing pipelines
|
26 |
+
3. Perform sliding window inference on CT volumes
|
27 |
+
4. Save the segmentation results
|
28 |
+
|
29 |
+
The model segments 118 different anatomical structures from CT scans.
|
30 |
+
|
31 |
+
## Setup
|
32 |
+
Install requirements and import necessary packages
|
33 |
+
|
34 |
+
|
35 |
+
```python
|
36 |
+
# Install lighter_zoo package
|
37 |
+
%pip install lighter_zoo -U -qq
|
38 |
+
```
|
39 |
+
|
40 |
+
Note: you may need to restart the kernel to use updated packages.
|
41 |
+
|
42 |
+
|
43 |
+
|
44 |
+
```python
|
45 |
+
# Imports
|
46 |
+
import torch
|
47 |
+
from lighter_zoo import SegResNet
|
48 |
+
from monai.transforms import (
|
49 |
+
Compose, LoadImage, EnsureType, Orientation,
|
50 |
+
ScaleIntensityRange, CropForeground, Invert,
|
51 |
+
Activations, AsDiscrete, KeepLargestConnectedComponent,
|
52 |
+
SaveImage
|
53 |
+
)
|
54 |
+
from monai.inferers import SlidingWindowInferer
|
55 |
+
```
|
56 |
+
|
57 |
+
Note: you may need to restart the kernel to use updated packages.
|
58 |
+
|
59 |
+
|
60 |
+
## Load Model
|
61 |
+
Download and initialize the pre-trained model from HuggingFace Hub
|
62 |
+
|
63 |
+
|
64 |
+
```python
|
65 |
+
# Load pre-trained model
|
66 |
+
model = SegResNet.from_pretrained(
|
67 |
+
"project-lighter/whole_body_segmentation",
|
68 |
+
force_download=True
|
69 |
+
)
|
70 |
+
```
|
71 |
+
|
72 |
+
|
73 |
+
config.json: 0%| | 0.00/162 [00:00<?, ?B/s]
|
74 |
+
|
75 |
+
|
76 |
+
|
77 |
+
model.safetensors: 0%| | 0.00/349M [00:00<?, ?B/s]
|
78 |
+
|
79 |
+
|
80 |
+
## Configure Inference
|
81 |
+
Set up sliding window inference for processing large volumes
|
82 |
+
|
83 |
+
|
84 |
+
```python
|
85 |
+
# Configure sliding window inference
|
86 |
+
inferer = SlidingWindowInferer(
|
87 |
+
roi_size=[96, 160, 160], # Size of patches to process
|
88 |
+
sw_batch_size=2, # Number of windows to process in parallel
|
89 |
+
overlap=0.625, # Overlap between windows (reduces boundary artifacts)
|
90 |
+
mode="gaussian" # Gaussian weighting for overlap regions
|
91 |
+
)
|
92 |
+
```
|
93 |
+
|
94 |
+
## Setup Processing Pipelines
|
95 |
+
Define preprocessing and postprocessing transforms
|
96 |
+
|
97 |
+
|
98 |
+
```python
|
99 |
+
# Preprocessing pipeline
|
100 |
+
preprocess = Compose([
|
101 |
+
LoadImage(ensure_channel_first=True), # Load image and ensure channel dimension
|
102 |
+
EnsureType(), # Ensure correct data type
|
103 |
+
Orientation(axcodes="SPL"), # Standardize orientation
|
104 |
+
# Scale intensity to [0,1] range, clipping outliers
|
105 |
+
ScaleIntensityRange(
|
106 |
+
a_min=-1024, # Min HU value
|
107 |
+
a_max=2048, # Max HU value
|
108 |
+
b_min=0, # Target min
|
109 |
+
b_max=1, # Target max
|
110 |
+
clip=True # Clip values outside range
|
111 |
+
),
|
112 |
+
CropForeground() # Remove background to reduce computation
|
113 |
+
])
|
114 |
+
|
115 |
+
# Postprocessing pipeline
|
116 |
+
postprocess = Compose([
|
117 |
+
Activations(softmax=True), # Apply softmax to get probabilities
|
118 |
+
AsDiscrete(argmax=True, dtype=torch.int32), # Convert to class labels
|
119 |
+
KeepLargestConnectedComponent(), # Remove small disconnected regions
|
120 |
+
Invert(transform=preprocess), # Restore original space
|
121 |
+
# Save the result
|
122 |
+
SaveImage(output_dir="./segmentations")
|
123 |
+
])
|
124 |
+
```
|
125 |
+
|
126 |
+
/home/suraj/miniconda3/lib/python3.10/site-packages/monai/utils/deprecate_utils.py:321: FutureWarning: monai.transforms.croppad.array CropForeground.__init__:allow_smaller: Current default value of argument `allow_smaller=True` has been deprecated since version 1.2. It will be changed to `allow_smaller=False` in version 1.5.
|
127 |
+
warn_deprecated(argname, msg, warning_category)
|
128 |
+
|
129 |
+
|
130 |
+
## Run Inference
|
131 |
+
Process an input CT scan and generate segmentation
|
132 |
+
|
133 |
+
|
134 |
+
```python
|
135 |
+
# Input path
|
136 |
+
input_path = "/home/suraj/Repositories/lighter-ct-fm/semantic-search-app/assets/scans/s0114.nii.gz"
|
137 |
+
|
138 |
+
# Preprocess input
|
139 |
+
input_tensor = preprocess(input_path)
|
140 |
+
|
141 |
+
# Run inference
|
142 |
+
with torch.no_grad():
|
143 |
+
output = inferer(input_tensor.unsqueeze(dim=0), model)[0]
|
144 |
+
|
145 |
+
# Copy metadata from input
|
146 |
+
output.applied_operations = input_tensor.applied_operations
|
147 |
+
output.affine = input_tensor.affine
|
148 |
+
|
149 |
+
# Postprocess and save result
|
150 |
+
result = postprocess(output[0])
|
151 |
+
print("✅ Segmentation completed and saved")
|
152 |
+
```
|
153 |
+
|
154 |
+
2025-01-16 18:41:57,674 INFO image_writer.py:197 - writing: /home/suraj/Repositories/lighter-ct-fm/semantic-search-app/assets/segmentations/0/0_trans.nii.gz
|
155 |
+
✅ Segmentation completed and saved
|
156 |
+
|
157 |
+
|
158 |
+
|