Commit Graph

2652 Commits

Author SHA1 Message Date
Alexandre Flament 2179079a91
[fix] fix flickr_noapi decoding (#1655)
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
2019-08-02 13:37:13 +02:00
cy8aer 4dc792e1e2 [enh] add invidious engine. (#1657)
closes #1372
2019-08-02 13:25:25 +02:00
Alexandre Flament 02bbbf59d6
[fix] Dockerfile: searx version stores in org.label-schema.version (#1659)
instead of org.label-schema.schema-version
2019-08-01 07:51:09 +02:00
Alexandre Flament 1bed39e6cb
Merge pull request #1658 from dalf/video-fixes
Fix dailymotion, google_videos and youtube_noapi engines
2019-08-01 07:44:30 +02:00
Dalf 0c032c8429 [fix] youtube_noapi engine: fix the title 2019-07-31 08:39:40 +02:00
Dalf 8b7ac56669 [fix] google_videos engine: some results don't a thumbnail 2019-07-31 08:38:45 +02:00
Dalf d44677e226 [fix] dailymotion engine: remove HTML tags from the description 2019-07-31 08:37:51 +02:00
Alexandre Flament 7f56c78876
Merge pull request #1650 from dalf/wikidata
[fix] wikidata engine: faster processing, remove one HTTP redirection.
2019-07-29 08:28:50 +02:00
Dalf 6e0285b2db [fix] wikidata engine: faster processing, remove one HTTP redirection.
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575)
2019-07-29 07:39:39 +02:00
Frank de Lange cbc5e13275 [enh] flickr_noapi: use complete JSON data block, add 'content', 'img_format', 'source', etc. (#1571)
Fetch complete JSON data block, use legend to extract images. 
Unquote urlencoded strings.
Add image description as 'content'. 
Add 'img_format' and 'source' data (needs PR #1567 to enable this data to be displayed). 
Show images which lack ownerid instead of discarding them.
2019-07-28 10:42:00 +02:00
Alexandre Flament c6ac39bcea
Merge pull request #1654 from MarcAbonce/small_fixes
[fix] Small fixes in Preferences view's text
2019-07-28 10:31:57 +02:00
Marc Abonce Seguin 59b1370824 [fix] small fixes in preferences view's text 2019-07-27 22:53:46 -07:00
Frank de Lange 204a2cbbf0 [fix] bing_videos (#1579)
use JSON where possible, compose 'content' using all available data, use correct 'url' (direct to source instead of redirect through bing)
2019-07-27 17:49:30 +02:00
Alexandre Flament 4b56a4bfaa
Merge pull request #1653 from dalf/bug-fixes
Various bug fixes
2019-07-27 09:11:03 +02:00
Marc Abonce Seguin e1bb0e33f2 [fix] escape perecent signs in translation file (#1528) 2019-07-27 09:01:02 +02:00
Dalf 70a01e5f02 [fix] don't merge image results if the img_src is different 2019-07-27 08:52:30 +02:00
Dalf 23611897ec [fix] make sure then engine name is lower case
Minor fix: "%s engine initialized" display the right engine name
2019-07-27 08:52:30 +02:00
Dalf 8645ef717e [fix] settings.yml: make sure all engine names are lower case 2019-07-27 08:52:30 +02:00
Dalf 7d8000d965 [fix] "!wp !wp test" send only one request to wikipedia.
Note that "!general !wp test" is not fixed because the category for "!wp" is "none".
2019-07-27 08:52:30 +02:00
Dalf 2fab23ab9a [fix] fix the debug message "starting webserver on ip:port"
was "port:ip"
2019-07-27 08:52:30 +02:00
Frank de Lange 11fc9913e9 [enh] bing_images: use data from embedded JSON to improve results (e.g. real page title) (#1568)
use data from embedded JSON to improve results (e.g. real page title), add image format and source info (see PR #1567), improve paging logic (it now works)
2019-07-27 08:22:02 +02:00
Alexandre Flament f34b5cedb1
[fix] fixes google play engines (#1651)
update commit 87baa74a86
2019-07-25 09:31:47 +02:00
Jonas Zohren cac8e7c6b8 [enh] add npm search engine (#1510) 2019-07-25 08:54:26 +02:00
volth eb182df132 [mod] restore btdigg engine as btdig.com (#1515) 2019-07-25 08:40:48 +02:00
rachmadani haryono 3b1122c5fa [fix] fix duden engine (#1594) 2019-07-25 08:17:45 +02:00
Venca24 87baa74a86 [fix] fixes google play engines and adds thumbnails to their results (#1612)
fix google play apps, google play apps, google play music engines

xpath engine: thumbnail_xpath can define an optional thumbnail
2019-07-25 07:46:41 +02:00
Alexandre Flament d0dd296424
Merge pull request #1642 from dalf/start-speed
Speed up start time and decrease CPU load
2019-07-23 13:18:29 +02:00
Dalf da0ce5880f [fix] fix soundcloud engine, speed up searx start time 2019-07-19 08:59:11 +02:00
Dalf 7e201cbf65 [mod] use cache in _match_language function to speed up searx start time significantly 2019-07-19 08:58:08 +02:00
Alexandre Flament 554a21e1d0
[enh] Add Server-Timing header (#1637)
Server Timing specification: https://www.w3.org/TR/server-timing/

In the browser Dev Tools, focus on the main request, there are the responses per engine in the Timing tab.
2019-07-17 10:38:45 +02:00
Alexandre Flament cfcbc3a5c3
[mod] settings.yml: update comments about the morty key. (#1639)
Fix #1310
Since commit af77ec35d9bd28facdab645a3d57ae340d2b501c Morty accepts base64 encoded key.
2019-07-17 10:29:28 +02:00
rachmadani haryono ec88fb8a0f [fix] secret_key can be bytes instead of a string (#1602)
Fix #1600
In settings.yml, the secret_key can be written as string or as base64 encoded data using !!binary notation.
2019-07-17 10:09:09 +02:00
rachmadani haryono 8f44014627 [fix] preference query parameter decoding (#1599)
Fix issue #1598
2019-07-17 09:42:40 +02:00
Alexandre Flament 582f3bfcbc
Merge pull request #1638 from dalf/suggestions
Suggestions: use RawTextQuery to make the suggestions URLs.
2019-07-17 09:20:07 +02:00
Dalf c7220b40ea Suggestions: use RawTextQuery to make the suggestions URLs. Update all themes accordingly. 2019-07-17 09:05:29 +02:00
Alexandre Flament ddee4861ce
Merge pull request #1630 from MarcAbonce/bang_fixes
[fix] Small fixes with bangs in queries
2019-07-16 11:08:36 +02:00
Alexandre Flament 4d38b8bef7
Merge branch 'master' into bang_fixes 2019-07-16 11:03:52 +02:00
Alexandre Flament f750df8713
Merge pull request #1636 from MarcAbonce/babel_update
Add Interlingua translation (fix #978)
2019-07-16 10:48:41 +02:00
Marc Abonce Seguin a3368c07a7 [enh] Add Interlingua locale (requires requirements update) 2019-07-15 13:38:51 -05:00
Alexandre Flament 2b5a7169a3
Merge pull request #1635 from asciimoo/update-documentation
README.rst: installation: add searx/searx-docker
2019-07-13 17:56:17 +02:00
Alexandre Flament efc2a87e77 README.rst: installation: add searx/searx-docker 2019-07-13 17:28:05 +02:00
Alexandre Flament 5b50e116b8
Merge pull request #1629 from dalf/docker
Update Docker image
2019-07-13 16:37:39 +02:00
Dalf 45702b77ca embedded iframe (youtube, dailymotion, vimeo): use https 2019-07-13 15:49:59 +02:00
Dalf 609ac5795a .travis.yml: robot tests: remove the xvfb dependency 2019-07-13 15:49:59 +02:00
Dalf fbe40001d3 Update Docker image
See #1561 , use uwsgi and Alpine Linux

Volume:
/var/log/uwsgi contains error log for 2 days (file uwsgi.log)
/etc/searx contains the settings.yml and uwsgi.ini files.
The docker image creates them if they don't exist.
The two files can be modified after the first run. See below.

Environement variables:
MORTY_URL : external URL of Morty
MORTY_KEY : base64 encoded key
BASE_URL : external URL of Searx
BIND_ADDRESS : internal HTTP port to listen to

Labels : org.label-schema.schema.*

Parameters:
-h : display this help
-d : will update the settings and quit immediately (settings.yml and uwsgi.ini)
-f : always update the settings (previous version saved with suffix .old).
     without this parameter, the new settings are copied with suffix .new

When the Docker image contains newer settings:
- without -f parameter: the new versions are copied to /etc/searx/settings.yml.new and /etc/searx/uwsgi.ini.new.
- with -f parameter:  the old versions are renamed with .old suffix. The new version replaces /etc/searx/settings.yml and /etc/searx/uwsgi.ini

Build using "./manage.sh docker_build", add "push" as parameter also push the Docker image.
The script requires a git repository to work (it makes sure that the last git tag matches searx/version.py)
"git describe" is used to create a meaningful version.
Example : 0.15.0-90-49c5bcb4-dirty (dirty means that the docker image was made with uncommited changes).

Use "docker inspect -f {{.Config.Labels.version}} searx" to get the version of an existing image.

.dockerignore based on .gitignore

.travis.yml: include docker stage
2019-07-13 15:49:59 +02:00
Alexandre Flament 3b8b862d8c
Merge branch 'master' into bang_fixes 2019-07-06 18:48:03 +02:00
Alexandre Flament 3faaf34e25
Fix ./manage.sh robot_tests (#1633)
doi_resolvers / default_doi_resolver were missing in the settings_robots.yml file, so the test server was not able to start (crash). Since the output wasn't displayed, it was not obvious why the Selenium couldn't connect to searx.
2019-07-06 18:47:35 +02:00
Marc Abonce Seguin 6626f01365 [fix] preserve bangs in suggestions and corrections 2019-07-01 19:52:34 -05:00
Marc Abonce Seguin 40272b0044 [fix] never pass bangs to autocomplete suggestions 2019-07-01 17:16:02 -05:00
Noémi Ványi 69c8ad86f3
Merge pull request #1614 from Nachtalb/ne/improve-infinite-scroll
Improve infinite scroll (and with it fixing a bug)
2019-06-30 13:49:16 +02:00