Falcon

License

FALCON 180B TII LICENSE VERSION 1.0

September 2023 falconllm.tii.ae

INTRODUCTORY NOTE This license is, in part, based on the Apache License Version 2.0 (available at http://www.apache.org/licenses/), with a series of modifications. The contribution of the Apache License 2.0 to the framing of this document is acknowledged. Please read this license carefully, as it is different to other ‘open access’ licenses you may have encountered previously. Use of Falcon180B for hosted services may require a separate license. TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. “License” shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 to 12 of this document. “Licensor” shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. “Legal Entity” shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, “control” means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. “You” (or “Your”) shall mean an individual or Legal Entity exercising permissions granted by this License. “Source” form shall mean the preferred form for making modifications, including but not limited to software source code, training datasets used for training or fine tuning a machine learning model or artificial intelligence model, documentation source, and configuration files. “Object” form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, a trained and/or fine-tuned machine learning model or artificial intelligence model, generated documentation, and conversions to other media types. “Work” shall mean the work of authorship, which in relation to the initial release of Falcon 180B is in Object form only, but in the case of any and all Derivative Works means that Work whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). “Derivative Works” shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. “Contribution” shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, “submitted” means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as “Not a Contribution.” “Contributor” shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. "Acceptable Use Policy” means the latest version from time to time of the policy designated as such hosted at FalconLLM.tii.ae. “Falcon 180B” shall mean TII’s 180 billion parameter Falcon large language model, initially made available in Object form only under this license at FalconLLM.tii.ae. "Hosting Application Address” means Falconllm.partnerships@tii.ae. “Hosting Use” has the meaning given in section 9 below. “Hosting User” means someone who has applied to make Hosting Use of the Work and been granted permission by the Licensor to make such Hosting Use subject to a separate licence agreement. “TII” shall mean the Technology Innovation Institute – Sole Proprietorship L.L.C., or any party nominated in writing by Technology Innovation Institute – Sole Proprietorship L.L.C. as its successor for the purposes of this License, or any party nominated in writing to be a successor to any successor for the purposes of this license.

1. Grant of Copyright License. 2.1. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 2.2. Other than where you are a Hosting User in accordance with Section 9, Your copyright license to use the Work shall be royalty free and without charge.

2. Grant of Patent License. 3.1. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 3.2. Other than where you are a Hosting User in accordance with Section 9, Your patent license to use the Work shall be royalty free and without charge.

3. Redistribution. 4.1. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) use-based restrictions incorporating the Acceptable Use Policy in the manner set out in Section 5 of this license, and which do not otherwise conflict with the Acceptable Use Policy, must be included as enforceable provisions by You in any type of legal agreement (e.g. a license) governing the use and/or distribution of the Work or any Derivative Works that You distribute; (b) hosting-based restrictions as set out in Section 8 of this license, and which do not otherwise conflict with those provisions, must be included as enforceable provisions by You in any type of legal agreement (e.g. a license) governing the use and/or distribution of the Work or any Derivative Works that You distribute; (c) You must give any other recipients of the Work or Derivative Works a copy of this License; and (d) You must cause any modified files to carry prominent notices stating that You changed the files; and (e) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (f) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. 4.2. Subject to Section 6, You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.

4. Acceptable Use 5.1. Subject to Section 5.3, your use of the Work or any Derivative Work must comply with the Acceptable Use Policy at all times. You shall procure that all persons using the Work or Derivative Work for you or on your behalf comply with the Acceptable Use Policy in their use. 5.2. You may not use the Work or any Derivative Work or any output from the Work or Derivative Work, whether directly or indirectly, to create other works for any purpose which conflicts with the Acceptable Use Policy. 5.3. The Acceptable Use Policy may be updated from time to time. You should monitor the web address at which the Acceptable Use Policy is hosted to ensure that your use of the Work or any Derivative Work complies with the updated Acceptable Use Policy.

5. Publication 6.1. You shall include prominently in any public statement regarding a Derivative Work the following statement: “[name of relevant Derivative Work] is built using Falcon LLM technology from the Technology Innovation Institute”. 6.2. You may request from TII a reasonably adjusted version of the above statement to suit the publication the relevant statement is being made in. TII will not unreasonably withhold or delay approval of such a request.

6. Submission of Contributions. 7.1. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. 7.2. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.

7. Trademarks. 8.1. Except as required for compliance with Section 6 of this License, this License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.

8. Hosting Use 9.1. Subject to section 9.2, "Hosting Use” means any use of the Work or a Derivative Work to offer shared instances or managed services based on the Work, any Derivative Work (including fine-tuned versions of a Work or Derivative Work) to third party users in an inference or finetuning API form. 9.2. The use of the Work or Derivative Works to provide applications and integrated end user products which use the Work or Derivative Work in the background shall not be considered Hosting Use. 9.3. Subject to Section 9.4, you are not licensed to use the Work or Derivative Work under this license for Hosting Use. Where You wish to make Hosting Use of Falcon 180B or any Work or Derivative Work, You must apply to TII for permission to make Hosting Use of that Work in writing via the Hosting Application Address, providing such information as may be required. 9.4. Where TII grants permission for You to make Hosting Use of the relevant Work, then for that purpose You shall be considered a Hosting User, and your use of Falcon 180B, the Work or Derivative Works shall be subject to the separate license granted by TII relating to that use.

9. Disclaimer of Warranty. 10.1. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.

10. Limitation of Liability. 11.1. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.

11. Accepting Warranty or Additional Liability. 12.1. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.

END OF TERMS AND CONDITIONS APPENDIX: How to apply the Falcon 180B TII License to your work. To apply the Falcon 180B TII License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Falcon 180B TII License, Version 1.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at FalconLLM.tii.ae. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

latest180b40b7binstructtext180b-chat180b-text180b-chat-q4_0180b-text-q4_040b-instruct40b-text40b-instruct-q4_040b-instruct-q4_140b-instruct-q5_040b-instruct-q5_140b-instruct-q8_040b-instruct-fp1640b-text-q4_040b-text-q4_140b-text-q5_040b-text-q5_140b-text-q8_040b-text-fp167b-instruct7b-text7b-instruct-q4_07b-instruct-q4_17b-instruct-q5_07b-instruct-q5_17b-instruct-q8_07b-instruct-fp167b-text-q4_07b-text-q4_17b-text-q5_07b-text-q5_17b-text-q8_07b-text-fp16

• 拉取模型

  ollama run falcon
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	4280f7257e73 · 4.2GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:180b
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 180B quantization 4-bit	e2bc879d7cee · 101GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:40b
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 4-bit	bc9368437a24 · 24GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:7b
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	4280f7257e73 · 4.2GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:instruct
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	4280f7257e73 · 4.2GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:text
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	e449cf4ba505 · 4.2GB

• 拉取模型

  ollama run falcon:180b-chat
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 180B quantization 4-bit	e2bc879d7cee · 101GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:180b-text
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 180B quantization 4-bit	f5905a53ed4b · 101GB

• 拉取模型

  ollama run falcon:180b-chat-q4_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 180B quantization 4-bit	e2bc879d7cee · 101GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:180b-text-q4_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 180B quantization 4-bit	f5905a53ed4b · 101GB

• 拉取模型

  ollama run falcon:40b-instruct
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 4-bit	bc9368437a24 · 24GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:40b-text
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 4-bit	b4137657e4e9 · 24GB

• 拉取模型

  ollama run falcon:40b-instruct-q4_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 4-bit	bc9368437a24 · 24GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:40b-instruct-q4_1
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 4-bit	9ec7eaf6cd59 · 26GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:40b-instruct-q5_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 5-bit	ca8da6223021 · 29GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:40b-instruct-q5_1
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 5-bit	9e4a62b9534b · 32GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:40b-instruct-q8_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 8-bit	6f9c09b99fc6 · 44GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:40b-instruct-fp16
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization F16	7cbd92dfea70 · 84GB
  template	{{- if and .First .System }} {{ .System }} {{- end }} User: {{ .Prompt }} Assistant:	84B

• 拉取模型

  ollama run falcon:40b-text-q4_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 4-bit	b4137657e4e9 · 24GB

• 拉取模型

  ollama run falcon:40b-text-q4_1
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 4-bit	d9b1df212f90 · 26GB

• 拉取模型

  ollama run falcon:40b-text-q5_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 5-bit	753deb72fbcd · 29GB

• 拉取模型

  ollama run falcon:40b-text-q5_1
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization 5-bit	10c176bb433f · 32GB

• 拉取模型

  ollama run falcon:40b-text-q8_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization F16	3573ccb06045 · 44GB

• 拉取模型

  ollama run falcon:40b-text-fp16
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 40B quantization F16	bc0d50593221 · 84GB

• 拉取模型

  ollama run falcon:7b-instruct
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	4280f7257e73 · 4.2GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:7b-text
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	e449cf4ba505 · 4.2GB

• 拉取模型

  ollama run falcon:7b-instruct-q4_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	4280f7257e73 · 4.2GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:7b-instruct-q4_1
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	f06a70f0b7d2 · 4.6GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:7b-instruct-q5_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 5-bit	ef910bf6af84 · 5.1GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:7b-instruct-q5_1
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 5-bit	878a7290ef26 · 5.5GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:7b-instruct-q8_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 8-bit	836fb3b71733 · 7.7GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:7b-instruct-fp16
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization F16	696dae724ad2 · 14GB
  params	{"stop":["User:","Assistant:"]}	31B
  template	{{ .System }} User: {{ .Prompt }} Assistant:	45B

• 拉取模型

  ollama run falcon:7b-text-q4_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	e449cf4ba505 · 4.2GB

• 拉取模型

  ollama run falcon:7b-text-q4_1
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 4-bit	51498c22efa8 · 4.6GB

• 拉取模型

  ollama run falcon:7b-text-q5_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 5-bit	539880f876a8 · 5.1GB

• 拉取模型

  ollama run falcon:7b-text-q5_1
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 5-bit	3a4e772d215e · 5.5GB

• 拉取模型

  ollama run falcon:7b-text-q8_0
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization 8-bit	9e4ddb71d35d · 7.7GB

• 拉取模型

  ollama run falcon:7b-text-fp16
  

• 模型信息 (model)

  Manifest	Info	Size
  model	arch falcon parameters 7B quantization F16	c2de45e2ed45 · 14GB

模型详情

Falcon LLM Falcon LLM 是 TII 的旗舰系列大型语言模型，从零开始使用自定义数据管道和分布式训练库构建。即将发布论文

为了促进合作并推动创新，我们已开源多个工件：

• 在 Falcon-180B TII 许可下的 Falcon-180B 预训练和聊天模型。Falcon-180B 是目前最大、最强大的开放访问模型。
• 在 Apache 2.0 软件许可下的 Falcon-7/40B 预训练和指导模型。Falcon-7B/40B 模型在其规模下是最先进的，超越了其他开源模型在 NLP 基准测试中的表现。
• RefinedWeb 数据集，一个具有严格过滤和大规模去重的庞大网络数据集，使仅使用网络数据训练的模型能够匹配或超越使用策划语料库训练的模型。有关更多信息，请参阅 📓 论文。RefinedWeb 在 ODC-By 1.0 下获得许可。

请查看以下 Falcon LLM 系列的详细工件列表：

Artefact	Link	Type	Details
🥇 Falcon-180B	Here	pretrained model	180B parameters trained on 3,500 billion tokens.
Falcon-180B-Chat	Here	chat model	Falcon-180B finetuned on a mixture of Ultrachat, Platypus and Airoboros.
🥈 Falcon-40B	Here	pretrained model	40B
Falcon-40B-Instruct	Here	instruction/chat model	Falcon-40B
🥉 Falcon-7B	Here	pretrained model	6.7B
Falcon-7B-Instruct	Here	instruction/chat model	Falcon-7B finetuned on the Baize, GPT4All, and GPTeacher datasets.
📀 RefinedWeb	Here	pretraining web dataset	~600 billion "high-quality" tokens.
Falcon-RW-1B	Here	pretrained model	1.3B parameters trained on 350 billion tokens.
Falcon-RW-7B	Here	pretrained model	7.5B parameters trained on 350 billion tokens.

Falcon-7BFalcon-40BFalcon-180B

🚀 Falcon-7B

Falcon-7B 是由 TII 构建的一个 7B 参数的因果解码器模型，只训练了 1500B 令牌的 RefinedWeb 并增强了策划语料库。它可在 Apache 2.0 许可下使用。

论文即将发布 😊。

🤗 要开始使用 Falcon（推理、微调、量化等），我们推荐阅读 这篇来自 HF 的精彩博客文章！

为什么使用 Falcon-7B？

• 它在比较的开源模型中表现优异（例如 MPT-7B, StableLM, RedPajama 等），这得益于在增强了策划语料库的 1500B 令牌的 RefinedWeb 上训练。查看 OpenLLM 排行榜。
• 其架构针对推理进行了优化，采用 FlashAttention（Dao et al., 2022）和多查询技术（Shazeer et al., 2019）。
• 它在允许商业使用的宽松 Apache 2.0 许可下提供，没有任何版税或限制。

⚠️ 这是一个原始的预训练模型，对于大多数用例应进一步进行微调。 如果您正在寻找一个更适合接受聊天格式通用指令的版本，我们推荐查看 Falcon-7B-Instruct。

🔥 寻找更强大的模型？ Falcon-40B 是 Falcon-7B 的大兄弟！

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = "tiiuae/falcon-7b"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
sequences = pipeline(
  "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
    max_length=200,
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")

💥 Falcon LLM 需要使用 PyTorch 2.0 与 transformers 一起使用！

想要快速推理 Falcon，请查看 文本生成推理！在这篇[博客文章]((https://huggingface.co/blog/falcon) 中阅读更多。

您将需要至少 16GB 的内存，以便快速运行 Falcon-7B 的推理。

Falcon-7B 模型卡片

模型详情

模型描述

• 开发者： https://www.tii.ae;
• 模型类型： 因果解码器仅模型;
• 语言（NLP）： 英语、德语、西班牙语、法语（以及在意大利语、葡萄牙语、波兰语、荷兰语、罗马尼亚语、捷克语、瑞典语中的有限能力）;
• 许可证： Apache 2.0.

模型来源

• 论文： 即将发布。

使用方式

直接使用

大语言模型研究；作为进一步专业化和微调的基础，针对特定用例（例如，摘要、文本生成、聊天机器人等）。

超出使用范围

未经充分评估风险和缓解措施即在生产中使用；任何可能被认为是不负责任或有害的用例。

偏见、风险和限制

Falcon-7B 只在英语和法语数据上进行训练，无法适当地推广到其他语言。此外，由于它是在代表网络的大规模语料库上训练的，它将携带在线上常见的刻板印象和偏见。

建议

我们建议 Falcon-7B 的用户考虑将其微调以适应感兴趣的特定任务集，并在任何生产使用中采取防护措施和适当的预防措施。

如何开始使用该模型

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = "tiiuae/falcon-7b"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
sequences = pipeline(
  "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
    max_length=200,
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")

训练详情

训练数据

Falcon-7B 在 RefinedWeb 的 1500B 令牌上进行了训练，这是一个高质量的过滤和去重的网络数据集，我们通过策划语料库进行了增强。我们策划的语料库的重要部分受到 The Pile（Gao et al., 2020）的启发。

数据来源	比例	令牌数	来源
RefinedWeb-English	79%	1185B	大规模网络爬虫
书籍	7%	110B
对话	6%	85B	Reddit, StackOverflow, HackerNews
代码	3%	45B
RefinedWeb-French	3%	45B	大规模网络爬虫
技术	2%	30B	arXiv, PubMed, USPTO 等。

数据使用 Falcon-7B/40B 分词器进行分词。

训练程序

Falcon-7B 在 384 个 A100 40GB GPUs 上进行训练，采用 2D 并行策略（PP=2, DP=192）结合 ZeRO。

训练超参数

超参数	值	备注
精度	bfloat16
优化器	AdamW
学习率	6e-4	40B 令牌预热，余弦衰减至 1.2e-5
权重衰减	1e-1
Z-loss	1e-4
批量大小	2304	30B 令牌递增

速度、大小、时间

训练发生在 2023 年 3 月初，历时约两周。

评估

论文即将发布。

请查看 OpenLLM 排行榜 以获取早期结果。

技术规格

模型架构和目标

Falcon-7B 是一个因果解码器仅模型，在因果语言建模任务上进行训练（即，预测下一个令牌）。

架构广泛地从 GPT-3 论文（Brown et al., 2020）中进行了调整，具有以下差异：

• 位置嵌入： 旋转（Su et al., 2021）；
• 注意力： 多查询（Shazeer et al., 2019）和 FlashAttention（Dao et al., 2022）；
• 解码器块： 并行注意力/MLP 与单层规范。

超参数	值	备注
层数	32
d_model	4544	为补偿多查询而增加
head_dim	64	为优化 FlashAttention 而减少
词汇量	65024
序列长度	2048

计算基础设施

硬件

Falcon-7B 在 AWS SageMaker 上的 384 个 A100 40GB GPU 的 P4d 实例上进行了训练。

软件

Falcon-7B 在自定义分布式训练代码库 Gigatron 上进行训练，使用 3D 并行方法结合 ZeRO 和高性能 Triton 核心（FlashAttention 等）。

引用

论文即将发布 😊。与此同时，您可以使用以下信息进行引用：

@article{falcon40b,
  title={{Falcon-40B}: an open large language model with state-of-the-art performance},
  author={Almazrouei, Ebtesam and Alobeidli, Hamza and Alshamsi, Abdulaziz and Cappelli, Alessandro and Cojocaru, Ruxandra and Debbah, Merouane and Goffinet, Etienne and Heslow, Daniel and Launay, Julien and Malartic, Quentin and Noune, Badreddine and Pannier, Baptiste and Penedo, Guilherme},
  year={2023}
}

要了解有关预训练数据集的更多信息，请查看 📓 RefinedWeb 论文。

@article{refinedweb,
  title={The {R}efined{W}eb dataset for {F}alcon {LLM}: outperforming curated corpora with web data, and web data only},
  author={Guilherme Penedo and Quentin Malartic and Daniel Hesslow and Ruxandra Cojocaru and Alessandro Cappelli and Hamza Alobeidli and Baptiste Pannier and Ebtesam Almazrouei and Julien Launay},
  journal={arXiv preprint arXiv:2306.01116},
  eprint={2306.01116},
  eprinttype = {arXiv},
  url={https://arxiv.org/abs/2306.01116},
  year={2023}
}

许可证

Falcon-7B 在 Apache 2.0 许可下提供。

联系方式

falconllm@tii.ae

🚀 Falcon-40B

Falcon-40B 是由 TII 构建的一个 40B 参数的因果解码器模型，训练于 1000B 令牌的 RefinedWeb 并增强了策划语料库。它可在 Apache 2.0 许可下使用。

论文即将发布 😊。

🤗 要开始使用 Falcon（推理、微调、量化等），我们推荐阅读 这篇来自 HF 的精彩博客文章！

为什么使用 Falcon-40B？

• 它是目前可用的最佳开源模型。 Falcon-40B 在性能上超过了 LLaMA、StableLM、RedPajama、MPT 等。参见 OpenLLM 排行榜。
• 其架构针对推理进行了优化，采用 FlashAttention（Dao et al., 2022）和多查询技术（Shazeer et al., 2019）。
• 它在允许商业使用的宽松 Apache 2.0 许可下提供，没有任何版税或限制。

⚠️ 这是一个原始的预训练模型，对于大多数用例应进一步进行微调。 如果您正在寻找一个更适合接受聊天格式通用指令的版本，我们推荐查看 Falcon-40B-Instruct。

💸 寻找一个更小、更经济的模型？ Falcon-7B 是 Falcon-40B 的小兄弟！

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = "tiiuae/falcon-40b"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
sequences = pipeline(
  "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
    max_length=200,
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")

💥 Falcon LLM 需要使用 PyTorch 2.0 与 transformers 一起使用！

想要快速推理 Falcon，请查看 文本生成推理！在这篇[博客文章]((https://huggingface.co/blog/falcon) 中阅读更多。

您将需要至少 85-100GB 的内存，以便快速运行 Falcon-40B 的推理。

Falcon-40B 模型卡片

模型详情

模型描述

• 开发者： https://www.tii.ae;
• 模型类型： 因果解码器仅模型;
• 语言（NLP）： 英语、德语、西班牙语、法语（以及在意大利语、葡萄牙语、波兰语、荷兰语、罗马尼亚语、捷克语、瑞典语中的有限能力）;
• 许可证： Apache 2.0 许可。

模型来源

• 论文： 即将发布。

使用方式

直接使用

大语言模型研究；作为进一步专业化和微调的基础，针对特定用例（例如，摘要、文本生成、聊天机器人等）。

超出使用范围

未经充分评估风险和缓解措施即在生产中使用；任何可能被认为是不负责任或有害的用例。

偏见、风险和限制

Falcon-40B 主要在英语、德语、西班牙语、法语上进行训练，同时在意大利语、葡萄牙语、波兰语、荷兰语、罗马尼亚语、捷克语、瑞典语上也有限的能力。它不会适当地推广到其他语言。此外，由于它是在代表网络的大规模语料库上训练的，它将携带在线上常见的刻板印象和偏见。

建议

我们建议 Falcon-40B 的用户考虑将其微调以适应感兴趣的特定任务集，并在任何生产使用中采取防护措施和适当的预防措施。

如何开始使用该模型

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = "tiiuae/falcon-40b"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
sequences = pipeline(
  "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
    max_length=200,
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")

训练详情

训练数据

Falcon-40B 在 RefinedWeb 的 1000B 令牌上进行了训练，这是一个高质量的过滤和去重的网络数据集，我们通过策划语料库进行了增强。我们策划的语料库的重要部分受到 The Pile（Gao et al., 2020）的启发。

数据来源	比例	令牌数	来源
RefinedWeb-English	75%	750B	大规模网络爬虫
RefinedWeb-Europe	7%	70B	欧洲大规模网络爬虫
书籍	6%	60B
对话	5%	50B	Reddit, StackOverflow, HackerNews
代码	5%	50B
技术	2%	20B	arXiv, PubMed, USPTO 等。

RefinedWeb-Europe 包含以下语言：

语言	多语言数据的比例	令牌数
德语	26%	18B
西班牙语	24%	17B
法语	23%	16B
意大利语	7%	5B
葡萄牙语	4%	3B
波兰语	4%	3B
荷兰语	4%	3B
罗马尼亚语	3%	2B
捷克语	3%	2B
瑞典语	2%	1B

数据使用 Falcon-7B/40B 分词器进行分词。

训练程序

Falcon-40B 在 384 个 A100 40GB GPUs 上进行训练，采用 3D 并行策略（TP=8, PP=4, DP=12）结合 ZeRO。

训练超参数

超参数	值	备注
精度	bfloat16
优化器	AdamW
学习率	1.85e-4	4B 令牌预热，余弦衰减至 1.85e-5
权重衰减	1e-1
Z-loss	1e-4
批量大小	1152	100B 令牌递增

速度、大小、时间

训练开始于 2022 年 12 月，历时两个月。

评估

论文即将发布。

请查看 OpenLLM 排行榜 以获取早期结果。

技术规格

模型架构和目标

Falcon-40B 是一个因果解码器仅模型，在因果语言建模任务上进行训练（即，预测下一个令牌）。

架构广泛地从 GPT-3 论文（Brown et al., 2020）中进行了调整，具有以下差异：

• 位置嵌入： 旋转（Su et al., 2021）；
• 注意力： 多查询（Shazeer et al., 2019）和 FlashAttention（Dao et al., 2022）；
• 解码器块： 并行注意力/MLP 与两层规范。

对于多查询，我们使用的是一个内部变种，它根据张量并行度独立使用键和值。

超参数	值	备注
层数	60
d_model	8192
head_dim	64	为优化 FlashAttention 而减少
词汇量	65024
序列长度	2048

计算基础设施

硬件

Falcon-40B 在 AWS SageMaker 上的 384 个 A100 40GB GPU 的 P4d 实例上进行了训练。

软件

Falcon-40B 在自定义分布式训练代码库 Gigatron 上进行训练，使用 3D 并行方法结合 ZeRO 和高性能 Triton 核心（FlashAttention 等）。

引用

论文即将发布 😊。在此期间，您可以使用以下信息进行引用：

@article{falcon40b,
  title={{Falcon-40B}: an open large language model with state-of-the-art performance},
  author={Almazrouei, Ebtesam and Alobeidli, Hamza and Alshamsi, Abdulaziz and Cappelli, Alessandro and Cojocaru, Ruxandra and Debbah, Merouane and Goffinet, Etienne and Heslow, Daniel and Launay, Julien and Malartic, Quentin and Noune, Badreddine and Pannier, Baptiste and Penedo, Guilherme},
  year={2023}
}

要了解有关预训练数据集的更多信息，请查看 📓 RefinedWeb 论文。

@article{refinedweb,
  title={The {R}efined{W}eb dataset for {F}alcon {LLM}: outperforming curated corpora with web data, and web data only},
  author={Guilherme Penedo and Quentin Malartic and Daniel Hesslow and Ruxandra Cojocaru and Alessandro Cappelli and Hamza Alobeidli and Baptiste Pannier and Ebtesam Almazrouei and Julien Launay},
  journal={arXiv preprint arXiv:2306.01116},
  eprint={2306.01116},
  eprinttype = {arXiv},
  url={https://arxiv.org/abs/2306.01116},
  year={2023}
}

许可证

Falcon-40B 在 Apache 2.0 许可下提供。

联系方式

falconllm@tii.ae

🚀 Falcon-180B

Falcon-180B 是由 TII 构建的一个 180B 参数的因果解码器模型，训练于 3500B 令牌的 RefinedWeb 并增强了策划语料库。它可在 Falcon-180B TII 许可 和 可接受使用政策 下使用。

论文即将发布 😊

🤗 要开始使用 Falcon（推理、微调、量化等），我们推荐阅读 HF 的这篇精彩博客文章 或者 40B 发布时的这篇文章！请注意，由于 180B 的规模超出了 transformers+acccelerate 能轻松处理的范围，我们推荐使用 文本生成推理。

您将需要至少 400GB 的内存，以便快速运行 Falcon-180B 的推理。

为什么使用 Falcon-180B？

• 它是目前可用的最佳开放访问模型，也是最好的模型之一。 Falcon-180B 在性能上超过了 LLaMA-2、StableLM、RedPajama、MPT 等。参见 OpenLLM 排行榜。
• 其架构针对推理进行了优化，采用多查询技术（Shazeer et al., 2019）。
• 它在允许商业使用的宽松许可下提供。
• ⚠️ 这是一个原始的预训练模型，对于大多数用例应进一步进行微调。 如果您正在寻找一个更适合接受聊天格式通用指令的版本，我们推荐查看 Falcon-180B-Chat。

💸 寻找一个更小、更经济的模型？ Falcon-7B 和 Falcon-40B 是 Falcon-180B 的小兄弟！

💥 Falcon LLM 需要使用 PyTorch 2.0 与 transformers 一起使用！

Falcon-180B 模型卡片

模型详情

模型描述

• 开发者： https://www.tii.ae;
• 模型类型： 因果解码器仅模型;
• 语言（NLP）： 英语、德语、西班牙语、法语（以及在意大利语、葡萄牙语、波兰语、荷兰语、罗马尼亚语、捷克语、瑞典语中的有限能力）;
• 许可证： Falcon-180B TII 许可 和 可接受使用政策。

模型来源

• 论文： 即将发布。

使用方式

参见 可接受使用政策。

直接使用

大语言模型研究；作为进一步专业化和微调的基础，针对特定用例（例如，摘要、文本生成、聊天机器人等）。

超出使用范围

未经充分评估风险和缓解措施即在生产中使用；任何可能被认为是不负责任或有害的用例。

偏见、风险和限制

Falcon-180B 主要在英语、德语、西班牙语、法语上进行训练，同时在意大利语、葡萄牙语、波兰语、荷兰语、罗马尼亚语、捷克语、瑞典语上也有限的能力。它不会适当地推广到其他语言。此外，由于它是在代表网络的大规模语料库上训练的，它将携带在线上常见的刻板印象和偏见。

建议

我们建议 Falcon-180B 的用户考虑将其微调以适应感兴趣的特定任务集，并在任何生产使用中采取防护措施和适当的预防措施。

如何开始使用该模型

要以完整的 bfloat16 精度运行模型推理，您需要大约 8x A100 80GB 或等效配置。

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = "tiiuae/falcon-180b"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
sequences = pipeline(
  "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
    max_length=200,
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
    print(f"Result: {seq['generated_text']}")

训练详情

训练数据

Falcon-180B 在 RefinedWeb 的 3500B 令牌上进行了训练，这是一个高质量的过滤和去重的网络数据集，我们通过策划语料库进行了增强。我们策划的语料库的重要部分受到 The Pile（Gao et al., 2020）的启发。

数据来源	比例	令牌数	来源
RefinedWeb-English	75%	2625B	大规模网络爬虫
RefinedWeb-Europe	7%	245B	欧洲大规模网络爬虫
书籍	6%	210B
对话	5%	175B	Reddit, StackOverflow, HackerNews
代码	5%	175B
技术	2%	70B	arXiv, PubMed, USPTO 等。

RefinedWeb-Europe 包含以下语言：

语言	多语言数据的比例	令牌数
德语	26%	64B
西班牙语	24%	59B
法语	23%	56B
意大利语	7%	17B
葡萄牙语	4%	10B
波兰语	4%	10B
荷兰语	4%	10B
罗马尼亚语	3%	7B
捷克语	3%	7B
瑞典语	2%	5B

数据使用 Falcon 分词器进行分词。

训练程序

Falcon-180B 在多达 4096 个 A100 40GB GPUs 上进行训练，采用 3D 并行策略（TP=8, PP=8, DP=64）结合 ZeRO。

训练超参数

超参数	值	备注
精度	bfloat16
优化器	AdamW
学习率	1.25e-4	4B 令牌预热，余弦衰减至 1.25e-5
权重衰减	1e-1
Z-loss	1e-4
批量大小	2048	100B 令牌递增

速度、大小、时间

训练始于 2023 年初。

评估

论文即将发布。

请查看 OpenLLM 排行榜 以获取早期结果。

技术规格

模型架构和目标

Falcon-180B 是一个因果解码器仅模型，在因果语言建模任务上进行训练（即，预测下一个令牌）。

架构广泛地从 GPT-3 论文（Brown et al., 2020）中进行了调整，具有以下差异：

• 位置嵌入： 旋转（Su et al., 2021）；
• 注意力： 多查询（Shazeer et al., 2019）和 FlashAttention（Dao et al., 2022）；
• 解码器块： 并行注意力/MLP 与两层规范。

对于多查询，我们使用的是一个内部变种，它根据张量并行度独立使用键和值（所谓的多组）。

超参数	值	备注
层数	80
d_model	14848
head_dim	64	为优化 FlashAttention 而减少
词汇量	65024
序列长度	2048

计算基础设施

硬件

Falcon-180B 在 AWS SageMaker 上的多达 4096 个 A100 40GB GPU 的 P4d 实例上进行了训练。

软件

Falcon-180B 在自定义分布式训练代码库 Gigatron 上进行训练，使用 3D 并行方法结合 ZeRO 和高性能 Triton 核心（FlashAttention 等）。

引用

论文即将发布 😊（这次是真的）。在此期间，您可以使用以下信息进行引用：

@article{falcon,
  title={The Falcon Series of Language Models: Towards Open Frontier Models},
  author={Almazrouei, Ebtesam and Alobeidli, Hamza and Alshamsi, Abdulaziz and Cappelli, Alessandro and Cojocaru, Ruxandra and Alhammadi, Maitha and Daniele, Mazzotta and Heslow, Daniel and Launay, Julien and Malartic, Quentin and Noune, Badreddine and Pannier, Baptiste and Penedo, Guilherme},
  year={2023}
}

要了解有关预训练数据集的更多信息，请查看 📓 RefinedWeb 论文。

@article{refinedweb,
  title={The {R}efined{W}eb dataset for {F}alcon {LLM}: outperforming curated corpora with web data, and web data only},
  author={Guilherme Penedo and Quentin Malartic and Daniel Hesslow and Ruxandra Cojocaru and Alessandro Cappelli and Hamza Alobeidli and Baptiste Pannier and Ebtesam Almazrouei and Julien Launay},
  journal={arXiv preprint arXiv:2306.01116},
  eprint={2306.01116},
  eprinttype = {arXiv},
  url={https://arxiv.org/abs/2306.01116},
  year={2023}
}

联系方式

falconllm@tii.ae