Squid url redirector

Check-in [d6f041d57d]
anonymous

Check-in [d6f041d57d]

Overview
Comment:limit log row size, add OPTIONS support
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | master | trunk
Files: files | file ages | folders
SHA3-256: d6f041d57d1cf32ac7d339dee4ad7bafdf9ff75227e384091b7d101d08c99c47
User & Date: arcade on 2018-10-28 15:23:18.023
Other Links: branch diff | manifest | tags
Context
2018-10-28
15:23
limit log row size, add OPTIONS support Leaf check-in: d6f041d57d user: arcade tags: master, trunk
2013-11-28
14:45
gevent 1.0 welcome... check-in: 9d7d80e594 user: arcade@b1t.name tags: master, trunk
Changes
Side-by-Side Diff Ignore Whitespace Patch
Modified squid-tagger.py from [31ac372562] to [e1f1188a8c].
︙
166
167
168
169
170
171
172
173

174
175
176
177
178
179
180

166
167
168
169
170
171
172

173
174
175
176
177
178
179
180








-
+








			if len(self._tail) > 0:
				rows[0] = self._tail + rows[0]
			# popping out last (incomplete) element
			self._tail = rows.pop(-1)
			# dropping all complete elements to the queue
			for row in rows:
				self.put_nowait(row)
				logger.info('< ' + row)
                                logger.info('< ' + row[0:1024])
		# sending EOF
		self.put_nowait(None)

stdin = FReadlineQueue(sys.stdin)

# wrapper against file handler that makes possible to queue some writes without stalling
︙
284
285
286
287
288
289
290
291

292
293
294
295
296
297
298

284
285
286
287
288
289
290

291
292
293
294
295
296
297
298








-
+








class Checker(object):
	__slots__ = frozenset(['_db', '_log', '_queue', '_request', '_stdout'])

	def __init__(self, queue, logger):
		self._db = tagDB()
		self._log = logger
		self._log.info('started')
		self._request = re.compile('^([0-9]+)\ ((http|ftp):\/\/)?([-\w.]+)(:[0-9]+)?(\/([^ ]*))?\ ([0-9.:]+)\/(-|[\w\.]+)\ (-|\w+)\ (-|GET|HEAD|POST|CONNECT).*$')
		self._request = re.compile('^([0-9]+)\ ((http|ftp):\/\/)?([-\w.]+)(:[0-9]+)?(\/([^ ]*))?\ ([0-9.:]+)\/(-|[\w.-]+)\ (-|\w+)\ (-|GET|HEAD|POST|CONNECT|OPTIONS).*$')
		self._queue = queue
		self._stdout = FWritelineQueue(sys.stdout)

	def process(self, id, site, ip_address, url_path, line = None):
		#self._log.info('trying {}'.format(site))
		result = self._db.check(site, ip_address)
		reply = None
︙