FLUX.1-dev FP8 Example Code Cleanup

Open HashingTag opened this issue 1 year ago • 1 comments

I was looking at the FLUX.1-dev FP8 example code in the documentation and noticed some unnecessary imports and variable declarations, which make it appear longer and cluttered. Here is the shorter version of the code I'm currently using. Please note that I'm not an expert, just trying to make it easier for anyone else looking to run the code.

import torch
from diffusers import FluxTransformer2DModel, FluxPipeline
from transformers import T5EncoderModel
from optimum.quanto import quantize, qfloat8, freeze

bfl_repo = "black-forest-labs/FLUX.1-dev"

transformer = FluxTransformer2DModel.from_single_file("https://huggingface.co/Kijai/flux-fp8/blob/main/flux1-dev-fp8.safetensors", torch_dtype=torch.bfloat16)
quantize(transformer, weights=qfloat8)
freeze(transformer)

text_encoder_2 = T5EncoderModel.from_pretrained(bfl_repo, subfolder="text_encoder_2", torch_dtype=torch.bfloat16)
quantize(text_encoder_2, weights=qfloat8)
freeze(text_encoder_2)

pipe = FluxPipeline.from_pretrained(bfl_repo, transformer=transformer, text_encoder_2=text_encoder_2, torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

prompt = "A cat holding a sign that says hello world"
image = pipe(
    prompt,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]

image.save("flux-dev-fp8.png")

Optimizations made: Removed unnecessary code and reduced the line count from 32 to 26.

Oct 16 '24 07:10 HashingTag

Thanks for making it leaner! Would you like to open a PR? You can tag @sayakpaul for reviews if you do

Oct 16 '24 11:10 a-r-r-o-w

how to load fp8 model use local_path? I don't want to re-download model again, I just replace url-model-path to my-localpath, error: OSError: black-forest-labs/FLUX.1-dev does not appear to have a file named config.json.

Nov 01 '24 02:11 zhaowendao30

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Nov 25 '24 15:11 github-actions[bot]

how to load fp8 model use local_path? I don't want to re-download model again, I just replace url-model-path to my-localpath, error: OSError: black-forest-labs/FLUX.1-dev does not appear to have a file named config.json.

You need to follow the documentation https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux#single-file-loading-for-the-fluxtransformer2dmodel.

Nov 25 '24 15:11 sayakpaul

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Dec 20 '24 15:12 github-actions[bot]

I tried out the code snippet mentioned in this issue and the doc example as well, I keep getting this same error. Could you please let me know if I should try out a different approach?

cc: @a-r-r-o-w

ImportError                               Traceback (most recent call last)
Cell In[2]ImportError                               Traceback (most recent call last)
Cell In[2], [line 20](vscode-notebook-cell:?execution_count=2&line=20)
     [17](vscode-notebook-cell:?execution_count=2&line=17) pipe.enable_model_cpu_offload()
     [19](vscode-notebook-cell:?execution_count=2&line=19) prompt = "A cat holding a sign that says hello world"
---> [20](vscode-notebook-cell:?execution_count=2&line=20) image = pipe(
     [21](vscode-notebook-cell:?execution_count=2&line=21)     prompt,
     [22](vscode-notebook-cell:?execution_count=2&line=22)     generator=torch.Generator("cpu").manual_seed(0)
     [23](vscode-notebook-cell:?execution_count=2&line=23) ).images[0]
     [25](vscode-notebook-cell:?execution_count=2&line=25) image.save("flux-dev-fp8.png")

File ~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:116, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    [113](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:113) @functools.wraps(func)
    [114](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:114) def decorate_context(*args, **kwargs):
    [115](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:115)     with ctx_factory():
--> [116](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:116)         return func(*args, **kwargs)

File ~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:783, in FluxPipeline.__call__(self, prompt, prompt_2, negative_prompt, negative_prompt_2, true_cfg_scale, height, width, num_inference_steps, sigmas, guidance_scale, num_images_per_prompt, generator, latents, prompt_embeds, pooled_prompt_embeds, ip_adapter_image, ip_adapter_image_embeds, negative_ip_adapter_image, negative_ip_adapter_image_embeds, negative_prompt_embeds, negative_pooled_prompt_embeds, output_type, return_dict, joint_attention_kwargs, callback_on_step_end, callback_on_step_end_tensor_inputs, max_sequence_length)
    [775](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:775) lora_scale = (
    [776](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:776)     self.joint_attention_kwargs.get("scale", None) if self.joint_attention_kwargs is not None else None
    [777](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:777) )
    [778](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:778) do_true_cfg = true_cfg_scale > 1 and negative_prompt is not None
    [779](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:779) (
    [780](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:780)     prompt_embeds,
    [781](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:781)     pooled_prompt_embeds,
    [782](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:782)     text_ids,
--> [783](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:783) ) = self.encode_prompt(
    [784](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:784)     prompt=prompt,
    [785](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:785)     prompt_2=prompt_2,
    [786](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:786)     prompt_embeds=prompt_embeds,
    [787](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:787)     pooled_prompt_embeds=pooled_prompt_embeds,
    [788](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:788)     device=device,
    [789](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:789)     num_images_per_prompt=num_images_per_prompt,
    [790](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:790)     max_sequence_length=max_sequence_length,
    [791](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:791)     lora_scale=lora_scale,
    [792](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:792) )
    [793](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:793) if do_true_cfg:
    [794](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:794)     (
    [795](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:795)         negative_prompt_embeds,
    [796](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:796)         negative_pooled_prompt_embeds,
   (...)
    [806](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:806)         lora_scale=lora_scale,
    [807](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:807)     )

File ~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:370, in FluxPipeline.encode_prompt(self, prompt, prompt_2, device, num_images_per_prompt, prompt_embeds, pooled_prompt_embeds, max_sequence_length, lora_scale)
    [364](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:364)     # We only use the pooled prompt output from the CLIPTextModel
    [365](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:365)     pooled_prompt_embeds = self._get_clip_prompt_embeds(
    [366](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:366)         prompt=prompt,
    [367](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:367)         device=device,
    [368](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:368)         num_images_per_prompt=num_images_per_prompt,
    [369](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:369)     )
--> [370](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:370)     prompt_embeds = self._get_t5_prompt_embeds(
    [371](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:371)         prompt=prompt_2,
    [372](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:372)         num_images_per_prompt=num_images_per_prompt,
    [373](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:373)         max_sequence_length=max_sequence_length,
    [374](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:374)         device=device,
    [375](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:375)     )
    [377](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:377) if self.text_encoder is not None:
    [378](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:378)     if isinstance(self, FluxLoraLoaderMixin) and USE_PEFT_BACKEND:
    [379](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:379)         # Retrieve the original scale by scaling back the LoRA layers

File ~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:256, in FluxPipeline._get_t5_prompt_embeds(self, prompt, num_images_per_prompt, max_sequence_length, device, dtype)
    [250](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:250)     removed_text = self.tokenizer_2.batch_decode(untruncated_ids[:, self.tokenizer_max_length - 1 : -1])
    [251](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:251)     logger.warning(
    [252](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:252)         "The following part of your input was truncated because `max_sequence_length` is set to "
    [253](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:253)         f" {max_sequence_length} tokens: {removed_text}"
    [254](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:254)     )
--> [256](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:256) prompt_embeds = self.text_encoder_2(text_input_ids.to(device), output_hidden_states=False)[0]
    [258](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:258) dtype = self.text_encoder_2.dtype
    [259](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:259) prompt_embeds = prompt_embeds.to(dtype=dtype, device=device)

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1739, in Module._wrapped_call_impl(self, *args, **kwargs)
   [1737](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1737)     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   [1738](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1738) else:
-> [1739](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1739)     return self._call_impl(*args, **kwargs)

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1750, in Module._call_impl(self, *args, **kwargs)
   [1745](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1745) # If we don't have any hooks, we want to skip the rest of the logic in
   [1746](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1746) # this function, and just call forward.
   [1747](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1747) if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   [1748](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1748)         or _global_backward_pre_hooks or _global_backward_hooks
   [1749](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1749)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1750](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1750)     return forward_call(*args, **kwargs)
   [1752](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1752) result = None
   [1753](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1753) called_always_called_hooks = set()

File ~/example/lib/python3.10/site-packages/accelerate/hooks.py:171, in add_hook_to_module.<locals>.new_forward(module, *args, **kwargs)
    [170](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:170) def new_forward(module, *args, **kwargs):
--> [171](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:171)     args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs)
    [172](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:172)     if module._hf_hook.no_grad:
    [173](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:173)         with torch.no_grad():

File ~/example/lib/python3.10/site-packages/accelerate/hooks.py:722, in CpuOffload.pre_forward(self, module, *args, **kwargs)
    [720](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:720)     self.prev_module_hook.offload()
    [721](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:721)     clear_device_cache()
--> [722](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:722) module.to(self.execution_device)
    [723](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:723) return send_to_device(args, self.execution_device), send_to_device(kwargs, self.execution_device)

File ~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3162, in PreTrainedModel.to(self, *args, **kwargs)
   [3157](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3157)     if dtype_present_in_args:
   [3158](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3158)         raise ValueError(
   [3159](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3159)             "You cannot cast a GPTQ model in a new `dtype`. Make sure to load the model using `from_pretrained` using the desired"
   [3160](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3160)             " `dtype` by passing the correct `torch_dtype` argument."
   [3161](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3161)         )
-> [3162](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3162) return super().to(*args, **kwargs)

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1343, in Module.to(self, *args, **kwargs)
   [1340](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1340)         else:
   [1341](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1341)             raise
-> [1343](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1343) return self._apply(convert)

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903, in Module._apply(self, fn, recurse)
    [901](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:901) if recurse:
    [902](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:902)     for module in self.children():
--> [903](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903)         module._apply(fn)
    [905](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:905) def compute_should_use_set_data(tensor, tensor_applied):
    [906](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:906)     if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    [907](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:907)         # If the new tensor has compatible tensor type as the existing tensor,
    [908](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:908)         # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    [913](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:913)         # global flag to let the user control whether they want the future
    [914](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:914)         # behavior of overwriting the existing tensor or not.

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903, in Module._apply(self, fn, recurse)
    [901](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:901) if recurse:
    [902](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:902)     for module in self.children():
--> [903](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903)         module._apply(fn)
    [905](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:905) def compute_should_use_set_data(tensor, tensor_applied):
    [906](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:906)     if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    [907](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:907)         # If the new tensor has compatible tensor type as the existing tensor,
    [908](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:908)         # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    [913](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:913)         # global flag to let the user control whether they want the future
    [914](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:914)         # behavior of overwriting the existing tensor or not.

    [... skipping similar frames: Module._apply at line 903 (4 times)]

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903, in Module._apply(self, fn, recurse)
    [901](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:901) if recurse:
    [902](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:902)     for module in self.children():
--> [903](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903)         module._apply(fn)
    [905](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:905) def compute_should_use_set_data(tensor, tensor_applied):
    [906](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:906)     if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    [907](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:907)         # If the new tensor has compatible tensor type as the existing tensor,
    [908](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:908)         # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    [913](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:913)         # global flag to let the user control whether they want the future
    [914](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:914)         # behavior of overwriting the existing tensor or not.

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:930, in Module._apply(self, fn, recurse)
    [926](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:926) # Tensors stored in modules are graph leaves, and we don't want to
    [927](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:927) # track autograd history of `param_applied`, so we have to use
    [928](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:928) # `with torch.no_grad():`
    [929](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:929) with torch.no_grad():
--> [930](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:930)     param_applied = fn(param)
    [931](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:931) p_should_use_set_data = compute_should_use_set_data(param, param_applied)
    [933](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:933) # subclasses may have multiple child tensors so we need to use swap_tensors

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1329, in Module.to.<locals>.convert(t)
   [1322](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1322)     if convert_to_format is not None and t.dim() in (4, 5):
   [1323](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1323)         return t.to(
   [1324](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1324)             device,
   [1325](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1325)             dtype if t.is_floating_point() or t.is_complex() else None,
   [1326](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1326)             non_blocking,
   [1327](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1327)             memory_format=convert_to_format,
   [1328](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1328)         )
-> [1329](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1329)     return t.to(
   [1330](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1330)         device,
   [1331](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1331)         dtype if t.is_floating_point() or t.is_complex() else None,
   [1332](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1332)         non_blocking,
   [1333](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1333)     )
   [1334](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1334) except NotImplementedError as e:
   [1335](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1335)     if str(e) == "Cannot copy out of meta tensor; no data!":

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:274, in WeightQBytesTensor.__torch_function__(cls, func, types, args, kwargs)
    [272](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:272) # Defer to operations dispatcher
    [273](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:273) with torch._C.DisableTorchFunctionSubclass():
--> [274](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:274)     return func(*args, **kwargs)

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:300, in WeightQBytesTensor.__torch_dispatch__(cls, op, types, args, kwargs)
    [298](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:298)     out_data = op(t._data, device=device, **kwargs)
    [299](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:299)     out_scale = op(t._scale, device=device, **kwargs)
--> [300](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:300)     return WeightQBytesTensor.create(
    [301](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:301)         t.qtype,
    [302](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:302)         t.axis,
    [303](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:303)         t.size(),
    [304](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:304)         t.stride(),
    [305](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:305)         out_data,
    [306](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:306)         out_scale,
    [307](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:307)         activation_qtype=t.activation_qtype,
    [308](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:308)         requires_grad=t.requires_grad,
    [309](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:309)     )
    [310](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:310) elif op is torch.ops.aten.t and cls is WeightQBytesTensor:
    [311](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:311)     t = args[0]

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:141, in WeightQBytesTensor.create(qtype, axis, size, stride, data, scale, activation_qtype, requires_grad)
    [132](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:132)     out_features, in_features = size
    [133](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:133)     if (
    [134](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:134)         in_features >= 64
    [135](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:135)         and out_features >= 64
   (...)
    [139](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:139)         )
    [140](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:140)     ):
--> [141](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:141)         return MarlinF8QBytesTensor(qtype, axis, size, stride, data, scale, requires_grad)
    [143](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:143) return WeightQBytesTensor(qtype, axis, size, stride, data, scale, activation_qtype, requires_grad)

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:79, in MarlinF8QBytesTensor.__init__(self, qtype, axis, size, stride, data, scale, requires_grad)
     [76](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:76)     scale = scale.reshape((-1, len(scale_perm_single)))[:, scale_perm_single]
     [77](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:77)     scale = scale.reshape(-1, out_features).contiguous()
---> [79](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:79)     data_packed = MarlinF8PackedTensor.pack(data)  # pack fp8 data to in32, and apply marlier re-ordering.
     [80](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:80) else:
     [81](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:81)     # When freezing (`model.freeze()`), the data is already a MarlinF8PackedTensor and scale is already repacked.
     [82](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:82)     data_packed = data

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:183, in MarlinF8PackedTensor.pack(cls, tensor)
    [179](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:179) data_int32 = pack_fp8_as_int32(tensor.T)  # pack fp8 data to in32.
    [181](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:181) perm = torch.empty(0, dtype=torch.int, device=tensor.device)
--> [183](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:183) data_int32 = torch.ops.quanto.pack_fp8_marlin(
    [184](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:184)     b_q_weight=data_int32, perm=perm, size_k=in_features, size_n=out_features, num_bits=8
    [185](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:185) )
    [187](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:187) return cls(data_int32, size=tensor.size(), stride=tensor.stride())

File ~/example/lib/python3.10/site-packages/torch/_ops.py:1123, in OpOverloadPacket.__call__(self, *args, **kwargs)
   [1121](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/_ops.py:1121) if self._has_torchbind_op_overload and _must_dispatch_in_python(args, kwargs):
   [1122](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/_ops.py:1122)     return _call_overload_packet_from_python(self, args, kwargs)
-> [1123](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/_ops.py:1123) return self._op(*args, **(kwargs or {}))

File ~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/__init__.py:167, in gptq_marlin_repack(b_q_weight, perm, size_k, size_n, num_bits)
    [165](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/__init__.py:165) assert b_q_weight.dim() == 2
    [166](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/__init__.py:166) assert b_q_weight.dtype == torch.int32
--> [167](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/__init__.py:167) return ext.lib.gptq_marlin_repack(b_q_weight, perm, size_k, size_n, num_bits)

File ~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:44, in Extension.lib(self)
     [40](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:40)             warnings.warn(
     [41](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:41)                 f"{self.name} was compiled with pytorch {pytorch_build_version}, but {torch.__version__} is installed: it will be recompiled."
     [42](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:42)             )
     [43](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:43) os.makedirs(self.build_directory, exist_ok=True)
---> [44](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:44) self._lib = load(
     [45](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:45)     name=self.name,
     [46](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:46)     sources=self.sources,
     [47](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:47)     extra_cflags=self.extra_cflags,
     [48](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:48)     extra_cuda_cflags=self.extra_cuda_cflags,
     [49](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:49)     build_directory=self.build_directory,
     [50](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:50) )
     [51](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:51) if not os.path.exists(version_file):
     [52](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:52)     with open(version_file, "w") as f:

File ~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1380, in load(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_python_module, is_standalone, keep_intermediates)
   [1288](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1288) def load(name,
   [1289](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1289)          sources: Union[str, List[str]],
   [1290](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1290)          extra_cflags=None,
   (...)
   [1298](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1298)          is_standalone=False,
   [1299](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1299)          keep_intermediates=True):
   [1300](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1300)     """
   [1301](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1301)     Load a PyTorch C++ extension just-in-time (JIT).
   [1302](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1302) 
   (...)
   [1378](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1378)         ...     verbose=True)
   [1379](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1379)     """
-> [1380](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1380)     return _jit_compile(
   [1381](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1381)         name,
   [1382](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1382)         [sources] if isinstance(sources, str) else sources,
   [1383](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1383)         extra_cflags,
   [1384](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1384)         extra_cuda_cflags,
   [1385](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1385)         extra_ldflags,
   [1386](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1386)         extra_include_paths,
   [1387](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1387)         build_directory or _get_build_directory(name, verbose),
   [1388](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1388)         verbose,
   [1389](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1389)         with_cuda,
   [1390](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1390)         is_python_module,
   [1391](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1391)         is_standalone,
   [1392](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1392)         keep_intermediates=keep_intermediates)

File ~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1823, in _jit_compile(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_python_module, is_standalone, keep_intermediates)
   [1820](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1820) if is_standalone:
   [1821](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1821)     return _get_exec_path(name, build_directory)
-> [1823](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1823) return _import_module_from_library(name, build_directory, is_python_module)

File ~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2245, in _import_module_from_library(module_name, path, is_python_module)
   [2243](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2243) spec = importlib.util.spec_from_file_location(module_name, filepath)
   [2244](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2244) assert spec is not None
-> [2245](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2245) module = importlib.util.module_from_spec(spec)
   [2246](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2246) assert isinstance(spec.loader, importlib.abc.Loader)
   [2247](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2247) spec.loader.exec_module(module)

File <frozen importlib._bootstrap>:571, in module_from_spec(spec)

File <frozen importlib._bootstrap_external>:1176, in create_module(self, spec)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

ImportError: /teamspace/studios/this_studio/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/build/quanto_cuda.so: cannot open shared object file: No such file or directory, [line 20](vscode-notebook-cell:?execution_count=2&line=20)
     [17](vscode-notebook-cell:?execution_count=2&line=17) pipe.enable_model_cpu_offload()
     [19](vscode-notebook-cell:?execution_count=2&line=19) prompt = "A cat holding a sign that says hello world"
---> [20](vscode-notebook-cell:?execution_count=2&line=20) image = pipe(
     [21](vscode-notebook-cell:?execution_count=2&line=21)     prompt,
     [22](vscode-notebook-cell:?execution_count=2&line=22)     generator=torch.Generator("cpu").manual_seed(0)
     [23](vscode-notebook-cell:?execution_count=2&line=23) ).images[0]
     [25](vscode-notebook-cell:?execution_count=2&line=25) image.save("flux-dev-fp8.png")

File ~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:116, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    [113](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:113) @functools.wraps(func)
    [114](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:114) def decorate_context(*args, **kwargs):
    [115](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:115)     with ctx_factory():
--> [116](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/_contextlib.py:116)         return func(*args, **kwargs)

File ~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:783, in FluxPipeline.__call__(self, prompt, prompt_2, negative_prompt, negative_prompt_2, true_cfg_scale, height, width, num_inference_steps, sigmas, guidance_scale, num_images_per_prompt, generator, latents, prompt_embeds, pooled_prompt_embeds, ip_adapter_image, ip_adapter_image_embeds, negative_ip_adapter_image, negative_ip_adapter_image_embeds, negative_prompt_embeds, negative_pooled_prompt_embeds, output_type, return_dict, joint_attention_kwargs, callback_on_step_end, callback_on_step_end_tensor_inputs, max_sequence_length)
    [775](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:775) lora_scale = (
    [776](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:776)     self.joint_attention_kwargs.get("scale", None) if self.joint_attention_kwargs is not None else None
    [777](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:777) )
    [778](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:778) do_true_cfg = true_cfg_scale > 1 and negative_prompt is not None
    [779](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:779) (
    [780](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:780)     prompt_embeds,
    [781](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:781)     pooled_prompt_embeds,
    [782](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:782)     text_ids,
--> [783](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:783) ) = self.encode_prompt(
    [784](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:784)     prompt=prompt,
    [785](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:785)     prompt_2=prompt_2,
    [786](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:786)     prompt_embeds=prompt_embeds,
    [787](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:787)     pooled_prompt_embeds=pooled_prompt_embeds,
    [788](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:788)     device=device,
    [789](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:789)     num_images_per_prompt=num_images_per_prompt,
    [790](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:790)     max_sequence_length=max_sequence_length,
    [791](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:791)     lora_scale=lora_scale,
    [792](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:792) )
    [793](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:793) if do_true_cfg:
    [794](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:794)     (
    [795](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:795)         negative_prompt_embeds,
    [796](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:796)         negative_pooled_prompt_embeds,
   (...)
    [806](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:806)         lora_scale=lora_scale,
    [807](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:807)     )

File ~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:370, in FluxPipeline.encode_prompt(self, prompt, prompt_2, device, num_images_per_prompt, prompt_embeds, pooled_prompt_embeds, max_sequence_length, lora_scale)
    [364](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:364)     # We only use the pooled prompt output from the CLIPTextModel
    [365](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:365)     pooled_prompt_embeds = self._get_clip_prompt_embeds(
    [366](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:366)         prompt=prompt,
    [367](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:367)         device=device,
    [368](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:368)         num_images_per_prompt=num_images_per_prompt,
    [369](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:369)     )
--> [370](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:370)     prompt_embeds = self._get_t5_prompt_embeds(
    [371](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:371)         prompt=prompt_2,
    [372](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:372)         num_images_per_prompt=num_images_per_prompt,
    [373](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:373)         max_sequence_length=max_sequence_length,
    [374](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:374)         device=device,
    [375](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:375)     )
    [377](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:377) if self.text_encoder is not None:
    [378](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:378)     if isinstance(self, FluxLoraLoaderMixin) and USE_PEFT_BACKEND:
    [379](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:379)         # Retrieve the original scale by scaling back the LoRA layers

File ~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:256, in FluxPipeline._get_t5_prompt_embeds(self, prompt, num_images_per_prompt, max_sequence_length, device, dtype)
    [250](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:250)     removed_text = self.tokenizer_2.batch_decode(untruncated_ids[:, self.tokenizer_max_length - 1 : -1])
    [251](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:251)     logger.warning(
    [252](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:252)         "The following part of your input was truncated because `max_sequence_length` is set to "
    [253](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:253)         f" {max_sequence_length} tokens: {removed_text}"
    [254](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:254)     )
--> [256](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:256) prompt_embeds = self.text_encoder_2(text_input_ids.to(device), output_hidden_states=False)[0]
    [258](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:258) dtype = self.text_encoder_2.dtype
    [259](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux.py:259) prompt_embeds = prompt_embeds.to(dtype=dtype, device=device)

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1739, in Module._wrapped_call_impl(self, *args, **kwargs)
   [1737](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1737)     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   [1738](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1738) else:
-> [1739](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1739)     return self._call_impl(*args, **kwargs)

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1750, in Module._call_impl(self, *args, **kwargs)
   [1745](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1745) # If we don't have any hooks, we want to skip the rest of the logic in
   [1746](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1746) # this function, and just call forward.
   [1747](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1747) if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   [1748](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1748)         or _global_backward_pre_hooks or _global_backward_hooks
   [1749](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1749)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1750](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1750)     return forward_call(*args, **kwargs)
   [1752](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1752) result = None
   [1753](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1753) called_always_called_hooks = set()

File ~/example/lib/python3.10/site-packages/accelerate/hooks.py:171, in add_hook_to_module.<locals>.new_forward(module, *args, **kwargs)
    [170](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:170) def new_forward(module, *args, **kwargs):
--> [171](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:171)     args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs)
    [172](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:172)     if module._hf_hook.no_grad:
    [173](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:173)         with torch.no_grad():

File ~/example/lib/python3.10/site-packages/accelerate/hooks.py:722, in CpuOffload.pre_forward(self, module, *args, **kwargs)
    [720](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:720)     self.prev_module_hook.offload()
    [721](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:721)     clear_device_cache()
--> [722](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:722) module.to(self.execution_device)
    [723](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/accelerate/hooks.py:723) return send_to_device(args, self.execution_device), send_to_device(kwargs, self.execution_device)

File ~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3162, in PreTrainedModel.to(self, *args, **kwargs)
   [3157](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3157)     if dtype_present_in_args:
   [3158](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3158)         raise ValueError(
   [3159](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3159)             "You cannot cast a GPTQ model in a new `dtype`. Make sure to load the model using `from_pretrained` using the desired"
   [3160](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3160)             " `dtype` by passing the correct `torch_dtype` argument."
   [3161](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3161)         )
-> [3162](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/transformers/modeling_utils.py:3162) return super().to(*args, **kwargs)

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1343, in Module.to(self, *args, **kwargs)
   [1340](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1340)         else:
   [1341](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1341)             raise
-> [1343](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1343) return self._apply(convert)

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903, in Module._apply(self, fn, recurse)
    [901](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:901) if recurse:
    [902](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:902)     for module in self.children():
--> [903](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903)         module._apply(fn)
    [905](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:905) def compute_should_use_set_data(tensor, tensor_applied):
    [906](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:906)     if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    [907](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:907)         # If the new tensor has compatible tensor type as the existing tensor,
    [908](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:908)         # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    [913](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:913)         # global flag to let the user control whether they want the future
    [914](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:914)         # behavior of overwriting the existing tensor or not.

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903, in Module._apply(self, fn, recurse)
    [901](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:901) if recurse:
    [902](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:902)     for module in self.children():
--> [903](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903)         module._apply(fn)
    [905](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:905) def compute_should_use_set_data(tensor, tensor_applied):
    [906](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:906)     if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    [907](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:907)         # If the new tensor has compatible tensor type as the existing tensor,
    [908](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:908)         # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    [913](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:913)         # global flag to let the user control whether they want the future
    [914](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:914)         # behavior of overwriting the existing tensor or not.

    [... skipping similar frames: Module._apply at line 903 (4 times)]

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903, in Module._apply(self, fn, recurse)
    [901](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:901) if recurse:
    [902](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:902)     for module in self.children():
--> [903](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:903)         module._apply(fn)
    [905](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:905) def compute_should_use_set_data(tensor, tensor_applied):
    [906](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:906)     if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    [907](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:907)         # If the new tensor has compatible tensor type as the existing tensor,
    [908](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:908)         # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    [913](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:913)         # global flag to let the user control whether they want the future
    [914](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:914)         # behavior of overwriting the existing tensor or not.

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:930, in Module._apply(self, fn, recurse)
    [926](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:926) # Tensors stored in modules are graph leaves, and we don't want to
    [927](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:927) # track autograd history of `param_applied`, so we have to use
    [928](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:928) # `with torch.no_grad():`
    [929](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:929) with torch.no_grad():
--> [930](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:930)     param_applied = fn(param)
    [931](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:931) p_should_use_set_data = compute_should_use_set_data(param, param_applied)
    [933](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:933) # subclasses may have multiple child tensors so we need to use swap_tensors

File ~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1329, in Module.to.<locals>.convert(t)
   [1322](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1322)     if convert_to_format is not None and t.dim() in (4, 5):
   [1323](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1323)         return t.to(
   [1324](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1324)             device,
   [1325](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1325)             dtype if t.is_floating_point() or t.is_complex() else None,
   [1326](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1326)             non_blocking,
   [1327](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1327)             memory_format=convert_to_format,
   [1328](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1328)         )
-> [1329](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1329)     return t.to(
   [1330](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1330)         device,
   [1331](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1331)         dtype if t.is_floating_point() or t.is_complex() else None,
   [1332](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1332)         non_blocking,
   [1333](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1333)     )
   [1334](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1334) except NotImplementedError as e:
   [1335](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/nn/modules/module.py:1335)     if str(e) == "Cannot copy out of meta tensor; no data!":

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:274, in WeightQBytesTensor.__torch_function__(cls, func, types, args, kwargs)
    [272](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:272) # Defer to operations dispatcher
    [273](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:273) with torch._C.DisableTorchFunctionSubclass():
--> [274](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:274)     return func(*args, **kwargs)

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:300, in WeightQBytesTensor.__torch_dispatch__(cls, op, types, args, kwargs)
    [298](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:298)     out_data = op(t._data, device=device, **kwargs)
    [299](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:299)     out_scale = op(t._scale, device=device, **kwargs)
--> [300](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:300)     return WeightQBytesTensor.create(
    [301](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:301)         t.qtype,
    [302](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:302)         t.axis,
    [303](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:303)         t.size(),
    [304](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:304)         t.stride(),
    [305](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:305)         out_data,
    [306](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:306)         out_scale,
    [307](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:307)         activation_qtype=t.activation_qtype,
    [308](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:308)         requires_grad=t.requires_grad,
    [309](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:309)     )
    [310](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:310) elif op is torch.ops.aten.t and cls is WeightQBytesTensor:
    [311](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:311)     t = args[0]

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:141, in WeightQBytesTensor.create(qtype, axis, size, stride, data, scale, activation_qtype, requires_grad)
    [132](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:132)     out_features, in_features = size
    [133](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:133)     if (
    [134](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:134)         in_features >= 64
    [135](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:135)         and out_features >= 64
   (...)
    [139](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:139)         )
    [140](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:140)     ):
--> [141](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:141)         return MarlinF8QBytesTensor(qtype, axis, size, stride, data, scale, requires_grad)
    [143](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/qbytes.py:143) return WeightQBytesTensor(qtype, axis, size, stride, data, scale, activation_qtype, requires_grad)

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:79, in MarlinF8QBytesTensor.__init__(self, qtype, axis, size, stride, data, scale, requires_grad)
     [76](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:76)     scale = scale.reshape((-1, len(scale_perm_single)))[:, scale_perm_single]
     [77](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:77)     scale = scale.reshape(-1, out_features).contiguous()
---> [79](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:79)     data_packed = MarlinF8PackedTensor.pack(data)  # pack fp8 data to in32, and apply marlier re-ordering.
     [80](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:80) else:
     [81](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:81)     # When freezing (`model.freeze()`), the data is already a MarlinF8PackedTensor and scale is already repacked.
     [82](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/qbits.py:82)     data_packed = data

File ~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:183, in MarlinF8PackedTensor.pack(cls, tensor)
    [179](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:179) data_int32 = pack_fp8_as_int32(tensor.T)  # pack fp8 data to in32.
    [181](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:181) perm = torch.empty(0, dtype=torch.int, device=tensor.device)
--> [183](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:183) data_int32 = torch.ops.quanto.pack_fp8_marlin(
    [184](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:184)     b_q_weight=data_int32, perm=perm, size_k=in_features, size_n=out_features, num_bits=8
    [185](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:185) )
    [187](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/tensor/weights/marlin/fp8/packed.py:187) return cls(data_int32, size=tensor.size(), stride=tensor.stride())

File ~/example/lib/python3.10/site-packages/torch/_ops.py:1123, in OpOverloadPacket.__call__(self, *args, **kwargs)
   [1121](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/_ops.py:1121) if self._has_torchbind_op_overload and _must_dispatch_in_python(args, kwargs):
   [1122](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/_ops.py:1122)     return _call_overload_packet_from_python(self, args, kwargs)
-> [1123](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/_ops.py:1123) return self._op(*args, **(kwargs or {}))

File ~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/__init__.py:167, in gptq_marlin_repack(b_q_weight, perm, size_k, size_n, num_bits)
    [165](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/__init__.py:165) assert b_q_weight.dim() == 2
    [166](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/__init__.py:166) assert b_q_weight.dtype == torch.int32
--> [167](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/__init__.py:167) return ext.lib.gptq_marlin_repack(b_q_weight, perm, size_k, size_n, num_bits)

File ~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:44, in Extension.lib(self)
     [40](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:40)             warnings.warn(
     [41](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:41)                 f"{self.name} was compiled with pytorch {pytorch_build_version}, but {torch.__version__} is installed: it will be recompiled."
     [42](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:42)             )
     [43](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:43) os.makedirs(self.build_directory, exist_ok=True)
---> [44](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:44) self._lib = load(
     [45](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:45)     name=self.name,
     [46](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:46)     sources=self.sources,
     [47](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:47)     extra_cflags=self.extra_cflags,
     [48](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:48)     extra_cuda_cflags=self.extra_cuda_cflags,
     [49](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:49)     build_directory=self.build_directory,
     [50](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:50) )
     [51](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:51) if not os.path.exists(version_file):
     [52](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/extension.py:52)     with open(version_file, "w") as f:

File ~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1380, in load(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_python_module, is_standalone, keep_intermediates)
   [1288](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1288) def load(name,
   [1289](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1289)          sources: Union[str, List[str]],
   [1290](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1290)          extra_cflags=None,
   (...)
   [1298](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1298)          is_standalone=False,
   [1299](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1299)          keep_intermediates=True):
   [1300](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1300)     """
   [1301](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1301)     Load a PyTorch C++ extension just-in-time (JIT).
   [1302](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1302) 
   (...)
   [1378](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1378)         ...     verbose=True)
   [1379](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1379)     """
-> [1380](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1380)     return _jit_compile(
   [1381](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1381)         name,
   [1382](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1382)         [sources] if isinstance(sources, str) else sources,
   [1383](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1383)         extra_cflags,
   [1384](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1384)         extra_cuda_cflags,
   [1385](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1385)         extra_ldflags,
   [1386](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1386)         extra_include_paths,
   [1387](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1387)         build_directory or _get_build_directory(name, verbose),
   [1388](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1388)         verbose,
   [1389](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1389)         with_cuda,
   [1390](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1390)         is_python_module,
   [1391](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1391)         is_standalone,
   [1392](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1392)         keep_intermediates=keep_intermediates)

File ~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1823, in _jit_compile(name, sources, extra_cflags, extra_cuda_cflags, extra_ldflags, extra_include_paths, build_directory, verbose, with_cuda, is_python_module, is_standalone, keep_intermediates)
   [1820](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1820) if is_standalone:
   [1821](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1821)     return _get_exec_path(name, build_directory)
-> [1823](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1823) return _import_module_from_library(name, build_directory, is_python_module)

File ~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2245, in _import_module_from_library(module_name, path, is_python_module)
   [2243](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2243) spec = importlib.util.spec_from_file_location(module_name, filepath)
   [2244](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2244) assert spec is not None
-> [2245](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2245) module = importlib.util.module_from_spec(spec)
   [2246](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2246) assert isinstance(spec.loader, importlib.abc.Loader)
   [2247](https://vscode-remote+vscode-002d01jnfy6vwr6d63dcqsnzdthj0v-002estudio-002elightning-002eai.vscode-resource.vscode-cdn.net/teamspace/studios/this_studio/example/~/example/lib/python3.10/site-packages/torch/utils/cpp_extension.py:2247) spec.loader.exec_module(module)

File <frozen importlib._bootstrap>:571, in module_from_spec(spec)

File <frozen importlib._bootstrap_external>:1176, in create_module(self, spec)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

ImportError: /teamspace/studios/this_studio/example/lib/python3.10/site-packages/optimum/quanto/library/extensions/cuda/build/quanto_cuda.so: cannot open shared object file: No such file or directory

Mar 04 '25 09:03 ParagEkbote

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Mar 28 '25 15:03 github-actions[bot]

Not Stale.

Mar 28 '25 15:03 ParagEkbote