Machine LearningFast Inference from Transformers via Speculative DecodingYaniv Leviathan, Matan Kalman, Yossi MatiasMachine LearningFast Inference from Transformers via Speculative DecodingYaniv Leviathan, Matan Kalman, Yossi MatiasSupplied bibliography entry.Read now
Machine LearningAccelerating Large Language Model Decoding with Speculative SamplingChen et al.Machine LearningAccelerating Large Language Model Decoding with Speculative SamplingChen et al.Supplied bibliography entry.Read now
Machine LearningLookahead: An Inference Acceleration FrameworkFu et al.Machine LearningLookahead: An Inference Acceleration FrameworkFu et al.Supplied bibliography entry.Read now
Machine LearningSpecInferXupeng Miao et al.Machine LearningSpecInferXupeng Miao et al.Supplied bibliography entry.Read now
Machine LearningGPTQElias Frantar et al.Machine LearningGPTQElias Frantar et al.Supplied bibliography entry.Read now
Machine LearningSmoothQuantXiao et al.Machine LearningSmoothQuantXiao et al.Supplied bibliography entry.Read now
Machine LearningLLM.int8()Tim Dettmers et al.Machine LearningLLM.int8()Tim Dettmers et al.Supplied bibliography entry.Read now
Machine LearningQLoRATim Dettmers et al.Machine LearningQLoRATim Dettmers et al.Supplied bibliography entry.Read now
Machine Learning8-bit OptimizersTim DettmersMachine Learning8-bit OptimizersTim DettmersSupplied bibliography entry.Read now
Machine LearningZeroQuantYao et al.Machine LearningZeroQuantYao et al.Supplied bibliography entry.Read now