From 369d6936d62ac7130e947e5c89f80bde6f79c850 Mon Sep 17 00:00:00 2001 From: terminaldweller Date: Mon, 4 Jul 2022 18:31:27 +0430 Subject: updated the readme --- README.md | 27 ++++----------------------- devourer/devourer.py | 2 ++ 2 files changed, 6 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index cb867f3..7f029c0 100644 --- a/README.md +++ b/README.md @@ -3,27 +3,8 @@ [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) # devourer -devourer is an api server that currently has the following endpoints and does the following things:
+Devourer is meant to be a sort of a knowledge aggregator though it has quite a long way to get there.
+Right now it can get text out of certain formats, summarize them or not and turn them into audio or not.
+There is a little feature that searches for a term on wikipedia and one that extracts requirements from RFCs.
-## /summ -```sh -https://localhost:19019/mila/summ?url=https://dilipkumar.medium.com/standalone-mongodb-on-kubernetes-cluster-19e7b5896b27&summary=newspaper&audio=true -``` -The `/summ` endpoint optionally summarizes the article and can also optionally send the article as an audio file.
-The parameters are `url`,`summary` tells the server which summarization method to use. the last parameter `audio` tells the server whether to just send the text result or an audio equivalent.
- -## /wiki -```sh -https://localhost:19019/mila/wiki?term=iommu&summary=newspaper&audio=true -``` -Searches wikipedia for the given `term` parameter. Like other endpoints, can optionally summarize the result and turn it into audio with `summary` and `audio` parameters.
- -## /reqs -```sh -https://localhost:19019/mila/reqs?url=https://www.ietf.org/rfc/rfc2865.txt&sourcetype=text -``` -Extracts the requirements from the contents inside a given url. The `sourcetype` parameter tells the server how to interpret the url contents. currently only `text` and `html` are supported as valid values.
- -## Usage Example - -Using FastAPI now so just check the docs. +To see what endpoints are present and how to use them or interact with them go to `/doc`. Devourer is powered by fastapi. Everything is in its default location.
diff --git a/devourer/devourer.py b/devourer/devourer.py index 21302b2..4d758e5 100644 --- a/devourer/devourer.py +++ b/devourer/devourer.py @@ -5,6 +5,7 @@ import contextlib import datetime import logging import os + # import re # import string import tempfile @@ -19,6 +20,7 @@ import rake_nltk # type:ignore import readability # type:ignore import refextract # type:ignore import requests # type:ignore + # import tika # type:ignore # import transformers from tika import parser as tparser # type:ignore -- cgit v1.2.3