Load / Initialization We will support a AutoModelForCausalLM (just same name as the one in transformers for now). The goal is to privide users a consitent way for from_pretrained method. Usage Option 1: Convert and Load Users will make a 2-line call for this way. Call convert_model to make a disk-t...

@shane-huang found that the current Linux .so files on sourceforge cannot be loaded with Glibc < 3.4.29 Env: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 Error message: Exception has occurred: RuntimeError (note: full exception trace is shown but execution is paused at: _run_module_as_main) Fai...

The pr #8244 didn't offer input format of convert_model. check the necessary elements of input_path Add input format description of convert_model make output_path a dir path for future compatability validate the input value of input_path and dtype a ./convert-model API

org.apache.spark:spark-core_2.12 need to be upgraded to avoid critical CVE issue. Need to support Spark3.4.0

Hi there! I was reading the deployment instructions for Kubernetes mode in https://github.com/intel-analytics/BigDL/blob/main/ppml/trusted-big-data-ml/scala/docker-occlum/README.md and noticed that in both Spark 3.1.3 Pi example client mode and Spark 3.1.3 Pi example cluster mode, only the spark-dri...

This issue is mainly proposing to collect the breaking changes between TF 2.11/12 and 2.10. @hkvision @lalalapotter Pre-issues related to the breaking changes: #7319 #7402 PR related: #7423 #8226 #8249

Expected to init a python PPMLContext in trusted bigdata container, but found dependencies missing: [email protected]:/ppml# python Python 3.8.10 (default, Mar 13 2023, 10:26:41) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from bigdl.ppml....

Issue As title, there are too many INFOs printed out in the training log, like 23-03-20 04:17:06 [task-result-getter-0] INFO TaskSetManager:57 - Finished task 198.0 in stage 58.0 (TID 2090) in 550 ms on 10.244.56.10 (executor 1) (199/201) 2023-03-20 04:17:06 INFO TaskSetManager:57 - Finished task...

I got this error when running bigdl in arm-based macOS. Exception in thread "main" java.lang.UnsatisfiedLinkError: /private/var/folders/8p/7jjj_ffx4q3g5n_zq447l6hh0000gn/T/dlNativeLoader8703271322043269346libiomp5.dylib: dlopen(/private/var/folders/8p/7jjj_ffx4q3g5n_zq447l6hh0000gn/T/dlNativeLoader8...

centos system python installation bigdl prompt error In addition, I use Git clone to directly install bash make-dist.sh. How should I use it after the installation is complete?