In jax>=0.3.16 I am getting this error when importing vit_jax: ImportError Traceback (most recent call last) /data/cristian/flax/docs/guides/transfer_learning.ipynb Cell 3 in <cell line: 1>() ----> 1 from vit_jax import models, checkpoint 2 from pathli...

It looks like flax.optim was removed here google/[email protected] and this is causing the colab notebook to not be working giving the error: No module named 'flax.optim' I think it needs to be updated to use Optax instead as mentioned here: https://github.com/google/flax/blob/main/docs/notebooks/optax_...

Hi, I'm trying to apply the pre-trained ViT to VTAB dataset but I have a problem preparing this dataset and modifying the code. Should I follow this to prepare the data? After that, should I modify the input_pipline.py? Could you pls give me some instructions on how to run ViT on VTAB? Thank you!

LiT models have disappeared from https://console.cloud.google.com/storage/browser/vit_models/lit We're working on making the available again asap.

Hi: Where can I get the pretrained model on cifar10&cifar100? like this pic accuracy model Thank you so much!! Wang-Y-S

There seems to be a problem. Is it normal to not have cls?

Is there code to train or fine tune LiT models somwhere? Notably, this colab just has code to use one of the existing models. https://colab.research.google.com/github/google-research/vision_transformer/blob/main/lit.ipynb

As described in Readme file, the ResNet, ViT, and MLP-Mixer checkpoints optimized using "Surrogate Gap Minimization Improves Sharpness-Aware Training" are provided. However, I can only find ViT-SAM, ViT-GSAM, MLP-Mixer-SAM, MLP-Mixer-GSAM checkpoints in gs://vit_models and gs://mixer_models, respect...

Virtual Machine: AWS EC2 g5.2xlarge, NVIDIA A10G GPU, 24G GPU memory When running vit_jax.ipynb, get_accuracy(params_repl) in the Evaluation section, I got the following error: INFO:absl:Load dataset info from ~/tensorflow_datasets/cifar10/3.0.2 0%| | 0/156 [00:00<?, ?it/s]2022-07-14 0...

With what mean and std do you use when normalizing the images right before pretraining?